SciΠvot
Harbor: pendingA 248-case benchmark evaluating scientific judgment by asking models to choose between original LLM plans and human-corrected alternatives using project records.
- Samples
- 248
- Released
- 2026
- Scores
- 5
RT-Safe
RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment
A simulated urban benchmark with 36 routes across 5 maps evaluating embodied agent safety under real-time constraints using VLMs.
How to run
ltb · rt-safe
$ npm i -g lettokenburn-cli$ ltb bench info rt-safe
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Sonnet | 2.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Sol | 2.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Astra | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| DeepSeek Chat | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Fable | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Grok | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Inkling | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
RT-Safe combines navigation tasks with moving actors, hazards, and traffic rules in Unreal Engine 5, allowing the world to evolve during inference. It records collisions and violations to evaluate safety, revealing that real-time execution drastically increases collision rates compared to static settings.