LetTokenBurn
Sign in
Auto57%Harbor: pendingAgentic / Tool Use

cua-speedrun

CUA-SpeedRun: Standardized Benchmarking of the Speed of Computer-Use Agents

A standardized platform evaluating computer-use agents (CUAs) on speed and cost efficiency using uniform VM infrastructure and representative task subsets across four benchmarks.

Paper

How to run

ltb · cua-speedrun
$ npm i -g lettokenburn-cli
$ ltb bench info cua-speedrun

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

8 sourced entries
HarnessSourceConfidenceLink
Gemini 3 Flash Preview79.6N/AIndependentmediumSep 30, 2026arxiv.org
Kimi K379.6N/AIndependentmediumSep 30, 2026arxiv.org
GPT-5.6 Luna81.6N/AIndependentmediumSep 30, 2026arxiv.org
GPT-5.6 Sol81.6N/AIndependentmediumSep 30, 2026arxiv.org
Gemini 3.7 Flash89.6N/AIndependentmediumSep 30, 2026arxiv.org
Claude Opus 591.6N/AIndependentmediumSep 30, 2026arxiv.org
Gemini 3.8 Flash91.6N/AIndependentmediumSep 30, 2026arxiv.org
GPT-6 Astra91.6N/AIndependentmediumSep 30, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

Content & Construction: cua-speedrun standardizes CUA evaluation by decoupling agent, benchmark, environment, and loop components. It utilizes a serverless cloud-based provider with identical virtual machine setups to ensure consistent wall-time measurements. The platform employs an energy minimization framework to select minimal representative task subsets that preserve statistical power while reducing evaluation time significantly (e.g., 84.9% reduction on OSWorld).

Scoring & Key Findings: Metrics include total task time, agent execution time vs environment operation time, and model call costs. The benchmark reveals unintuitive insights: increasing reasoning effort can reduce completion time by avoiding repeated failures, while faster I/O environments can sometimes slow agents down due to lack of training on varying speeds.