Model
google/gemma-4-31B-it
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AI Consilium | 2.4 | mean_judge_score | structured | Independent | medium | arxiv.org |
| CritPt | 1.4 | accuracy | N/A | Independent | low | epoch.ai |
| GPQA Diamond | 75.8 | accuracy | N/A | Independent | high | epoch.ai |
| HeRE | 0.9 | f1 | N/A | Independent | medium | arxiv.org |
| SimpleQA Verified | 10.4 | accuracy | N/A | Independent | high | epoch.ai |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AptMQL-Bench | 62.0 | soft-ex | N/A | Independent | medium | arxiv.org |
| SciCode | 43.4 | score | N/A | Independent | medium | artificialanalysis.ai |
| SolBehave | 92.7 | success_rate | N/A | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AgentHorizon | 46.8 | accuracy | opencode | Independent | medium | arxiv.org |
| CSEG-Bench | 0.3 | exact | N/A | Independent | medium | arxiv.org |
| HICO-Bench | 63.8 | success_rate | N/A | Independent | medium | arxiv.org |
| KiranaBench | -3730.0 | profit | N/A | Independent | medium | arxiv.org |
| PlanBench | 43.5 | accuracy | N/A | Independent | medium | arxiv.org |
| ReFract | 0.4 | success_rate | function calling | Independent | medium | arxiv.org |
| TiGanEval | 1.6 | success_rate | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| 3DVerifierBench | 62.6 | accuracy | N/A | Independent | medium | arxiv.org |
| DSV-Mem | 8.5 | accuracy | N/A | Independent | medium | arxiv.org |
| EgoGears | 56.4 | accuracy | N/A | Independent | medium | arxiv.org |
| SpaceConflict | 57.0 | accuracy | N/A | Independent | medium | arxiv.org |
| VisionQ | 57.0 | accuracy | N/A | Independent | medium | arxiv.org |
| VisualTrust | 83.9 | accuracy | N/A | Independent | medium | arxiv.org |
| VisualWorldBench | 47.4 | accuracy | N/A | Independent | medium | arxiv.org |
| WOVEN | 76.3 | accuracy | N/A | Independent | medium | arxiv.org |
Instruction Following
Instruction Following| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| CreativePreferences | 0.7 | auc | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| ImpactBench | 63.4 | impactbench_pos._(%) | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract ReasoningReproduced by the community
Loading community runs…