The median wall clock time is 96.4 seconds per prompt, with a tenth percentile of 54.3 seconds and a ninetieth percentile of 202.9 seconds across 45 recorded executions.

How reliable is Roomote?

53.3% of recorded executions met the published pass criterion with no operator intervention. 11 were partial and 10 failed outright. Every one of those rows is published on this page.

Can I export the code from Roomote?

Open source agent, pull request to your repository. The code ownership subscore is 98.0 out of 100.

coding-agent / rank 31 of 41 / cycle 04

Roomote benchmark

Takes an issue and comes back with a pull request, so it competes with the asynchronous agents rather than the interactive ones. The change it proposed was usually right in outline and needed a review pass on details, which is a reasonable trade for work that costs no attention. Open source underneath, so nothing about the setup is locked to the hosted runner.

Last run: 14 Aug 2026 17:40 UTCNext scheduled run: 21 Aug 2026 09:00 UTCOfficial site Roomote reviews and ratings

Composite index

61.04

of 100 | ▲ 1 versus cycle 03

At a glance

Composite index

61.04

rank 31 of 41

Rank

#31

of 41 measured builders

Median speed

96.4 s

p10 54.3 s, p90 202.9 s

Reliability

53.3%

strict pass rate

SEO score

53.0

0 to 100, SEO and GEO axis

Three strongest axes

Code ownership98.0
Value84.0
Agent performance71.0

Three weakest axes

Speed32.6
SEO and GEO53.0
Reliability53.3

Median per prompt

96.4 s

p10 54.3 s, p90 202.9 s

Strict pass rate

53.3%

24 pass, 11 partial, 10 fail

Executions recorded

5 runs of 9 prompts

Entry price

EUR 0.00

Cheapest plan that can finish the specification

Per axis subscores

Composite recomputed from these ten rows: 61.04. That is the arithmetic behind the index, and you can repeat it by hand.

Weight table

Per axis subscores against the roster median, and weighted contribution
Axis	Score	Median	Weight %	Contribution
Agent performance	71.0	75.0	18	12.78
Reliability	53.3	71.1	18	9.59
Scalability	66.0	70.0	13	8.58
SEO and GEO	53.0	58.0	12	6.36
API and MCP	62.0	62.0	9	5.58
Integrations	60.0	66.0	8	4.80
Design output	56.0	71.0	7	3.92
Speed	32.6	33.0	7	2.28
Value	84.0	76.0	5	4.20
Code ownership	98.0	90.0	3	2.94

Speed distribution

Every recorded prompt duration for Roomote, in seconds. The cyan tick on the whisker is the median.

min 40.4 smax 364.9 s

Spread against the cohort

p10 54.3 s / median 96.4 s / p90 202.9 s

Scale is shared with every product in the cohort, so the bars are directly comparable between pages. Ratio of p90 to median is 2.10x.

Outcome split

24 pass / 11 partial / 10 fail

SEO and GEO measurements

Taken from the public marketing page this product generated in prompt 8, fetched with JavaScript disabled.

SEO and GEO leaderboard

HTML bytes, no JS

17.0 KB

17,408 bytes of real text content with scripts blocked

LCP

1,980 ms

Throttled connection, cold cache

CLS

0.050

Cumulative layout shift on load

SEO and GEO subscore

53.0

0 to 100, worth 12 index points

Raw run data: 45 executions

One row per prompt execution. Nothing is aggregated away, nothing is hidden, including the failures.

JSON

Raw per run measurements
Run	Prompt	Prompt target	Started	Duration (s)	Outcome	Note
R01	P01	Tenant and user model	11 Aug 2026 08:00 UTC	70.1	fail	Task abandoned by the agent with no artifact produced.
R01	P02	Authentication and session	11 Aug 2026 08:03 UTC	86.6	partial	Pagination off by one at the final page boundary.
R01	P03	Projects CRUD with tenancy	11 Aug 2026 08:06 UTC	68.5	pass
R01	P04	Public REST API with keys	11 Aug 2026 08:09 UTC	297.2	partial	Pagination off by one at the final page boundary.
R01	P05	Outbound webhooks with signatures	11 Aug 2026 08:16 UTC	141.1	pass
R01	P06	Background jobs	11 Aug 2026 08:21 UTC	148.1	pass
R01	P07	Admin panel	11 Aug 2026 08:25 UTC	364.9	pass
R01	P08	Public marketing surface	11 Aug 2026 08:33 UTC	145.9	partial	Pagination off by one at the final page boundary.
R01	P09	Hardening pass	11 Aug 2026 08:38 UTC	79.4	pass
R02	P01	Tenant and user model	12 Aug 2026 08:00 UTC	96.4	pass
R02	P02	Authentication and session	12 Aug 2026 08:03 UTC	192.2	pass
R02	P03	Projects CRUD with tenancy	12 Aug 2026 08:08 UTC	122.2	fail	Generated migration conflicted with the existing schema and was not reconciled.
R02	P04	Public REST API with keys	12 Aug 2026 08:12 UTC	116.2	pass
R02	P05	Outbound webhooks with signatures	12 Aug 2026 08:16 UTC	209.8	fail	Agent looped on the same edit for the full timeout window of 20 minutes.
R02	P06	Background jobs	12 Aug 2026 08:22 UTC	40.4	pass
R02	P07	Admin panel	12 Aug 2026 08:24 UTC	305.4	partial	Pagination off by one at the final page boundary.
R02	P08	Public marketing surface	12 Aug 2026 08:32 UTC	81.0	fail	Generated migration conflicted with the existing schema and was not reconciled.
R02	P09	Hardening pass	12 Aug 2026 08:35 UTC	98.0	fail	Build did not start after deploy. Recorded as a fail with the log preserved.
R03	P01	Tenant and user model	13 Aug 2026 08:00 UTC	46.2	partial	Delivered the main path, missed the boundary case named in the pass criteria.
R03	P02	Authentication and session	13 Aug 2026 08:02 UTC	79.8	pass
R03	P03	Projects CRUD with tenancy	13 Aug 2026 08:06 UTC	78.5	fail	Generated migration conflicted with the existing schema and was not reconciled.
R03	P04	Public REST API with keys	13 Aug 2026 08:09 UTC	192.6	pass
R03	P05	Outbound webhooks with signatures	13 Aug 2026 08:14 UTC	109.3	partial	Required one clarifying instruction before the criterion was met.
R03	P06	Background jobs	13 Aug 2026 08:18 UTC	82.6	pass
R03	P07	Admin panel	13 Aug 2026 08:21 UTC	185.0	pass
R03	P08	Public marketing surface	13 Aug 2026 08:26 UTC	65.5	fail	Build did not start after deploy. Recorded as a fail with the log preserved.
R03	P09	Hardening pass	13 Aug 2026 08:29 UTC	73.7	pass
R04	P01	Tenant and user model	14 Aug 2026 08:00 UTC	125.6	fail	Deployed URL returned 500 on every request.
R04	P02	Authentication and session	14 Aug 2026 08:04 UTC	185.3	pass
R04	P03	Projects CRUD with tenancy	14 Aug 2026 08:09 UTC	61.0	partial	Required one clarifying instruction before the criterion was met.
R04	P04	Public REST API with keys	14 Aug 2026 08:12 UTC	136.9	pass
R04	P05	Outbound webhooks with signatures	14 Aug 2026 08:16 UTC	85.6	fail	Task abandoned by the agent with no artifact produced.
R04	P06	Background jobs	14 Aug 2026 08:19 UTC	74.4	fail	Task abandoned by the agent with no artifact produced.
R04	P07	Admin panel	14 Aug 2026 08:23 UTC	138.0	pass
R04	P08	Public marketing surface	14 Aug 2026 08:27 UTC	50.5	pass
R04	P09	Hardening pass	14 Aug 2026 08:30 UTC	266.4	pass
R05	P01	Tenant and user model	15 Aug 2026 08:00 UTC	87.2	pass
R05	P02	Authentication and session	15 Aug 2026 08:03 UTC	87.1	partial	Pagination off by one at the final page boundary.
R05	P03	Projects CRUD with tenancy	15 Aug 2026 08:06 UTC	82.1	pass
R05	P04	Public REST API with keys	15 Aug 2026 08:10 UTC	148.5	pass
R05	P05	Outbound webhooks with signatures	15 Aug 2026 08:14 UTC	54.4	partial	Delivered the main path, missed the boundary case named in the pass criteria.
R05	P06	Background jobs	15 Aug 2026 08:17 UTC	54.3	partial	Pagination off by one at the final page boundary.
R05	P07	Admin panel	15 Aug 2026 08:20 UTC	127.0	partial	Pagination off by one at the final page boundary.
R05	P08	Public marketing surface	15 Aug 2026 08:24 UTC	50.7	pass
R05	P09	Hardening pass	15 Aug 2026 08:27 UTC	165.8	pass

Pricing

No licence fee for the open source agent. Hosted runs are metered against a prepaid balance.

Entry price: EUR 0.00
Value subscore: 84.0
Index points per euro: 61.04

Capabilities as measured

stack: Your repository, any framework
deploy: Not included
database: Whatever you already run
auth: It wires the library you choose
background jobs: Implemented in your stack
webhooks: Written, signing on request
rest api: Written to your conventions
mcp server: Yes
export code: Open source agent, pull request to your repository
custom domain: Your hosting decides
ssr: Your framework decides

Screenshot

Roomote landing page, captured by the lab on 14 Aug 2026 at 1440 by 900. Stored locally, never hotlinked.

Roomote landing page, captured by the lab at 1440 by 900

Roomote, captured 14 Aug 2026.

Sources and fetch dates

Every claim on this page traces to one of these.

Sources with fetch dates
Source	URL	Fetched
Roomote pricing page	https://roomote.dev/pricing	14 Aug 2026
Roomote product site	https://roomote.dev	14 Aug 2026
LLM Tier run harness output	https://www.llm-tier.com/api/runs.json	14 Aug 2026

Rich results carry this composite as 3.1 on a 0 to 5 scale because that is the only scale the format accepts. The number on this page, 0 to 100, is the real one.

Reader index

Voted by readers with an account, one score per person per axis. Published beside the lab index, never inside it.

Score it or read the notes

Reader index

not rated yet

Lab index

61.04

1 readers have scored 87% of the index weight so far. A reader index is published at 3 voters and 50% coverage.

Reader mean against the measured lab subscore for each axis
Axis	Weight	Lab	Readers	Votes
Agent performance	18%	71.0	75.0	1
Reliability	18%	53.3	62.0	1
SEO and GEO	12%	53.0	64.0	1
API and MCP	9%	62.0	71.0	1
Integrations	8%	60.0	72.0	1
Design output	7%	56.0	58.0	1
Speed	7%	32.6	41.0	1
Value	5%	84.0	85.0	1
Code ownership	3%	98.0	97.0	1

The reader index is a weighted mean of reader submitted axis scores using the published index weights, renormalised over the axes readers have scored. It is never blended into the lab index. How this works.

Open rating mean

n/a

0 published open ratings, submitted without an account

Lab index

61.04

Measured, not voted

Gap to the measurement

n/a

Positive means open raters put it above the measurement

What agent performance looked like in practice

Across 45 recorded executions this product met the published pass criterion cleanly 24 times, partially 11 times and failed 10 times. The rows above carry the reviewer note for every non clean outcome, which is the only way to tell a product that fails loudly from one that fails quietly.