Benchmarks
SWE-bench Verified
Models scored
35
Best score
0.83
Metric
mean_score
SWE-bench Verified is an AI evaluation tracked by Epoch AI, with 35 scored model versions on record. Epoch AI runs this evaluation directly, so the scores are reproducible against their published logs.
Scored models span November 2024 to June 2026. The best score recorded is 0.83, measured as mean_score. Models come from Z.ai (Zhipu AI), Alibaba, DeepSeek, Google, Moonshot, OpenAI, Anthropic, Google DeepMind.
Full record
- Score metric
- mean_score
- Models scored
- 35
- Best score recorded
- 0.83
- Earliest model scored
- Nov 20, 2024
- Most recent model scored
- Jun 16, 2026
- Third-party reported
- No
Leaderboard
| 01 | claude-opus-4-7_max | Anthropic | 0.83 |
| 02 | gpt-5.5-pre-release_xhigh | OpenAI | 0.81 |
| 03 | gemini-3.5-flash_high | 0.79 | |
| 04 | claude-opus-4-6 | Anthropic | 0.79 |
| 05 | glm-5.2_max | Z.ai (Zhipu AI) | 0.79 |
| 06 | deepseek-v4-pro_max | DeepSeek | 0.78 |
| 07 | qwen3.7-max | Alibaba | 0.77 |
| 08 | gpt-5.4-2026-03-05_high | OpenAI | 0.77 |
| 09 | claude-opus-4-5-20251101 | Anthropic | 0.77 |
| 10 | qwen3.6-max-preview | Alibaba | 0.77 |
| 11 | kimi-k2.6 | Moonshot | 0.77 |
| 12 | gemini-3.1-pro-preview-customtools | Google DeepMind | 0.76 |
| 13 | gemini-3-flash-preview | Google DeepMind | 0.75 |
| 14 | claude-sonnet-4-6 | Anthropic | 0.75 |
| 15 | gpt-5.3-codex_high | OpenAI | 0.75 |
| 16 | glm-5.1 | Z.ai (Zhipu AI) | 0.74 |
| 17 | kimi-k2.5 | Moonshot | 0.74 |
| 18 | gpt-5.2-2025-12-11_high | OpenAI | 0.74 |
| 19 | gpt-5-2025-08-07_high | OpenAI | 0.74 |
| 20 | claude-opus-4-1-20250805 | Anthropic | 0.73 |
| 21 | gemini-3-pro-preview | Google DeepMind | 0.73 |
| 22 | glm-5 | Z.ai (Zhipu AI) | 0.72 |
| 23 | gpt-5-2025-08-07_medium | OpenAI | 0.71 |
| 24 | claude-sonnet-4-5-20250929 | Anthropic | 0.71 |
| 25 | claude-opus-4-20250514 | Anthropic | 0.71 |
| 26 | gpt-5.1-2025-11-13_high | OpenAI | 0.68 |
| 27 | gpt-5-mini-2025-08-07_medium | OpenAI | 0.65 |
| 28 | o3-2025-04-16_medium | OpenAI | 0.62 |
| 29 | claude-3-7-sonnet-20250219 | Anthropic | 0.61 |
| 30 | qwen3.6-plus | Alibaba | 0.58 |
| 31 | gemini-2.5-pro | Google DeepMind | 0.58 |
| 32 | gpt-4.1-2025-04-14 | OpenAI | 0.49 |
| 33 | gpt-4o-2024-11-20 | OpenAI | 0.31 |
More benchmarks
SourceEpoch AI, 'AI Benchmarking Hub'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/benchmarks. Licensed under CC BY 4.0.
← All benchmarks