Model
GPT-4o
OpenAI · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Balrog | 32.3 | average_progress | N/A | Independent | medium | balrogai.com |
| ChronoCulture | 66.9 | both_periods_accurate_(%) | N/A | Independent | medium | arxiv.org |
| CritPt | 0.0 | accuracy | N/A | Independent | low | epoch.ai |
| EnigmaEval | 0.8 | accuracy | N/A | Independent | medium | labs.scale.com |
| GDPval | 9.9 | win_rate | N/A | Independent | low | epoch.ai |
| GPQA | 53.6 | accuracy | N/A | Independent | medium | arxiv.org |
| GPQA Diamond | 49.2 | accuracy | N/A | Independent | high | epoch.ai |
| HLE | 2.7 | accuracy | N/A | Independent | low | epoch.ai |
| LiveBench | 55.3 | global_average | N/A | Independent | medium | livebench.ai |
| MMLU | 88.1 | exact_match | N/A | Independent | medium | arxiv.org |
| MOF-VERIFY | 62.8 | macro-f1 | N/A | Independent | medium | arxiv.org |
| News QA | 70.2 | exact_match | N/A | Independent | medium | arxiv.org |
| PeerQA | 30.9 | macro_f1 | N/A | Independent | medium | arxiv.org |
| PHRBench | 72.1 | auroc | N/A | Independent | medium | arxiv.org |
| ScienceQA | 88.5 | score | N/A | Independent | medium | arxiv.org |
| SimpleQA Verified | 26.0 | accuracy | N/A | Independent | high | epoch.ai |
Math
Math| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| FrontierMath | 1.0 | accuracy | N/A | Independent | medium | paperswithcode.com |
| NLData2Opt | 0.4 | accuracy | N/A | Independent | medium | arxiv.org |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Abt-Buy | 92.2 | f1_(%) | N/A | Independent | medium | arxiv.org |
| Aider Polyglot | 23.1 | percent_correct | N/A | Independent | medium | aider.chat |
| Amazon-Google | 63.5 | f1_(%) | N/A | Independent | medium | arxiv.org |
| ArchitectureIQ | 40.0 | accuracy | architectureiq | Independent | medium | arxiv.org |
| GSO | 0.0 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| PLCWorld | 92.8 | success_rate | N/A | Independent | medium | arxiv.org |
| SWE-bench Verified | 31.0 | resolved_rate | N/A | Independent | high | epoch.ai |
| VHDL-REPOBENCH | 60.9 | bc_(pass@1) | N/A | Independent | medium | arxiv.org |
| Web-Bench | 17.2 | pass@1 | N/A | Independent | medium | arxiv.org |
| WebApp1k-Duo-React | 53.1 | pass@1 | N/A | Independent | medium | arxiv.org |
| WebApp1K-React | 88.5 | pass@1 | N/A | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| 6-DoF SpatialBench | 36.2 | total | N/A | Independent | medium | arxiv.org |
| EHR-RobustGym | 80.9 | success_rate | N/A | Independent | medium | arxiv.org |
| GeoNatureAgent (GNA) | 41.6 | accuracy | N/A | Independent | medium | arxiv.org |
| HICO-Bench | 51.3 | success_rate | N/A | Independent | medium | arxiv.org |
| KlinikeBench | 3.9 | strict_pass@1 | N/A | Independent | medium | arxiv.org |
| Talk2Agent | 58.5 | execution | N/A | Independent | medium | arxiv.org |
| TheAgentCompany | 8.6 | resolved_rate | N/A | Independent | medium | github.com |
| TrialPD-Bench | 41.7 | f1 | agent | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| ChemExplain | 68.1 | accuracy | N/A | Independent | medium | arxiv.org |
| DataVista | 57.1 | accuracy | N/A | Independent | medium | arxiv.org |
| FG-Sudoku | 0.3 | accuracy | N/A | Independent | medium | arxiv.org |
| GeoBench | 71.0 | acw_country_% | N/A | Independent | medium | geobench.org |
| HAPACT | 28.9 | [email protected] | N/A | Independent | medium | arxiv.org |
| Knossos | 41.2 | f1 | N/A | Independent | medium | arxiv.org |
| MYOPIA | 72.8 | accuracy | N/A | Independent | medium | arxiv.org |
| R-GroundBench | 69.3 | accuracy | N/A | Independent | medium | arxiv.org |
| ScenePercept-Bench | 0.6 | score | N/A | Independent | medium | arxiv.org |
| SocialNav-SUB | 0.3 | pa | N/A | Independent | medium | arxiv.org |
| Video-MME | 77.2 | accuracy_(%) | N/A | Independent | medium | arxiv.org |
| Video-MME | 71.9 | overall_(no_subtitles) | N/A | Independent | medium | epoch.ai |
| VideoDB's OCR Benchmark Public Collection | 76.2 | average_accuracy | N/A | Independent | medium | arxiv.org |
| Vinoground | 59.2 | text_score | N/A | Independent | medium | paperswithcode.com |
| Vinoground | 54.0 | text_score | N/A | Independent | medium | paperswithcode.com |
| Visual Sanad | 83.4 | discrimination | N/A | Independent | medium | arxiv.org |
| VLM2-Bench | 37.5 | gc-mat | N/A | Independent | medium | arxiv.org |
| Zero-shot Video Question Answering on LongVideoBench | 64.0 | accuracy_(%_) | N/A | Independent | medium | arxiv.org |
Long Context
Long ContextSafety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Cybench | 12.5 | unguided_%_solved | N/A | Independent | medium | cybench.github.io |
| Jailbreak Guardrails Benchmark | 40.0 | bypass_rate | N/A | Independent | medium | arxiv.org |
| LatentRiskBench | 0.9 | d_state | N/A | Independent | medium | arxiv.org |
| MADBench | 73.1 | amplification_factor | N/A | Independent | medium | arxiv.org |
| OMIT | 0.1 | obr | N/A | Independent | medium | arxiv.org |
| OpenMTB-Audit | 0.7 | accuracy | prompt-checklist | Independent | medium | arxiv.org |
| Sense and Sensitivity | 91.4 | accuracy | N/A | Independent | medium | arxiv.org |
| SSUI-Dialog | 70.4 | avg | N/A | Independent | medium | arxiv.org |
| ToolHACE | 65.8 | f1 | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract ReasoningMultilingual
MultilingualReproduced by the community
Loading community runs…