Benchmarks · Vals AI
MMLU Pro (Vals-run)
General knowledge benchmark: multiple-choice questions across 14 subjects, including law. Vals marks the leaderboard "Archived Benchmark": "Since performance on this benchmark has saturated, we no longer run this benchmark on new model releases" (last update 1 September 2026).
- Models measured
- 110
- Top
- 92.38 %Claude Fable 5.1within the standard error of ±0.27
- Version
- 1
- Measured
- 1 September 2026
01
Ranking
At the top: Top 15
Vals AI Legal Research Bench, all-pass [1]. Hatched bars are rejected by the compliance gate.
- 1Claude Fable 5.192.38%
- 2Claude Opus 591.59%
- 3Claude Fable 591.50%
- 4Gemini 3.1 Pro Preview (02/26)90.99%
- 5Gemini 3.8 Flash90.22%
- 6Gemini 3.7 Flash90.12%
- 7Gemini 3 Pro (11/25)90.10%
- 8Claude Opus 4.789.87%
- 9Claude Opus 4.889.58%
- 10Gemini 3.5 Flash89.52%
- 11Grok 4.689.40%
- 12Qwen 3.7 Max89.31%
- 13Gemini 3.6 Flash89.28%
- 14Grok 4.589.22%
- 15Claude Opus 4.6 (Thinking)89.11%
Source: Vals AI (MMLU Pro (Vals-run)) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.
| # Model | Score | Cost/task | Gate | PrudentScore |
|---|---|---|---|---|
| 1Claude Fable 5.1Anthropic | 92.38%± 0.27 | - | Rejected | no score |
| 2Claude Opus 5Anthropic | 91.59%± 0.28 | - | Passes | 72 |
| 3Claude Fable 5Anthropic | 91.50%± 0.28 | - | Rejected | no score |
| 4Gemini 3.1 Pro Preview (02/26)Google | 90.99%± 0.28 | - | Rejected | no score |
| 5Gemini 3.8 FlashGoogle | 90.22%± 0.29 | - | Passes | 62 |
| 6Gemini 3.7 FlashGoogle | 90.12%± 0.30 | - | Passes | 58 |
| 7Gemini 3 Pro (11/25)Google | 90.10%± 0.29 | - | Rejected | no score |
| 8Claude Opus 4.7Anthropic | 89.87%± 0.30 | - | Passes | 54 |
| 9Claude Opus 4.8Anthropic | 89.58%± 0.30 | - | Passes | 62 |
| 10Gemini 3.5 FlashGoogle | 89.52%± 0.31 | - | Passes | 47 |
| 11Grok 4.6xAI (SpaceXAI) | 89.40%± 0.30 | - | Rejected | no score |
| 12Qwen 3.7 MaxAlibaba | 89.31%± 0.30 | - | Not yet assessed | no score |
| 13Gemini 3.6 FlashGoogle | 89.28%± 0.30 | - | Passes | 46 |
| 14Grok 4.5xAI (SpaceXAI) | 89.22%± 0.31 | - | Rejected | no score |
| 15Claude Opus 4.6 (Thinking)Anthropic | 89.11%± 0.46 | - | Passes | no score |
| 16GPT-5.6 SolOpenAI | 89.10%± 0.31 | - | Passes | 57 |
| 17Muse Spark 1.1Meta | 88.73%± 0.32 | - | Rejected | no score |
| 18Qwen 3.8 MaxAlibaba | 88.60%± 0.31 | - | Not yet assessed | no score |
| 19Gemini 3 Flash (12/25)Google | 88.59%± 0.32 | - | Rejected | no score |
| 20Muse Spark 1.2Meta | 88.28%± 0.32 | - | Rejected | no score |
| 21GPT 5.5OpenAI | 88.14%± 0.32 | - | Passes | 51 |
| 22Kimi K3Moonshot AI | 87.97%± 0.32 | - | Rejected | no score |
| 23Claude Opus 4.1 (Thinking)Anthropic | 87.92%± 0.33 | - | Rejected | no score |
| 24Qwen 3.6 PlusAlibaba | 87.67%± 0.32 | - | Not yet assessed | no score |
| 25Kimi K2.6Moonshot AI | 87.57%± 0.33 | - | Rejected | no score |
| 26Claude Sonnet 5Anthropic | 87.55%± 0.37 | - | Passes | 59 |
| 27GPT 5.4 (xhigh)OpenAI | 87.48%± 0.42 | - | Passes | 0 |
| 28Claude Sonnet 4.5 (Thinking)Anthropic | 87.36%± 0.39 | - | Passes | no score |
| 29Claude Sonnet 4.6Anthropic | 87.34%± 0.43 | - | Passes | 55 |
| 30Muse SparkMeta | 87.32%± 0.33 | - | Rejected | no score |
| 31Claude Opus 4.5 (Thinking)Anthropic | 87.26%± 0.38 | - | Passes | no score |
| 32DeepSeek V4 Pro (2026-04-23)DeepSeek | 87.25%± 0.34 | - | Rejected | no score |
| 33Claude Opus 4.1 (Nonthinking)Anthropic | 87.21%± 0.33 | - | Rejected | no score |
| 34Qwen 3.5 PlusAlibaba | 87.18%± 0.33 | - | Not yet assessed | no score |
| 35MiniMax-M2.1MiniMax | 87.05%± 0.33 | - | Passes | no score |
| 36DeepSeek V4 Pro (0813)DeepSeek | 86.97%± 0.34 | - | Rejected | no score |
| 37GLM 5.1Z.ai | 86.90%± 0.33 | - | Rejected | no score |
| 38GLM 5.3Z.ai | 86.77%± 0.34 | - | Rejected | no score |
| 39GLM 5.2Z.ai | 86.71%± 0.33 | - | Rejected | no score |
| 40GPT-5.6 TerraOpenAI | 86.66%± 0.33 | - | Passes | 53 |
| 41GPT 5OpenAI | 86.54%± 0.34 | - | Passes | no score |
| 42GPT 5.1OpenAI | 86.38%± 0.34 | - | Passes | no score |
| 43InklingThinking Machines | 86.30%± 0.34 | - | Rejected | no score |
| 44Grok 4.20 (Reasoning)xAI (SpaceXAI) | 86.25%± 0.34 | - | Rejected | no score |
| 45Gemini 3.1 Flash Lite PreviewGoogle | 86.24%± 0.34 | - | Not yet assessed | no score |
| 46GPT 5.2OpenAI | 86.23%± 0.34 | - | Passes | no score |
| 47DeepSeek V4 Flash 0731DeepSeek | 86.21%± 0.34 | - | Rejected | no score |
| 48Claude Opus 4 (Nonthinking)Anthropic | 86.17%± 0.34 | - | Rejected | no score |
| 49GLM 5.3 FlashZ.ai | 86.06%± 0.34 | - | Rejected | no score |
| 50GPT-5.6 LunaOpenAI | 86.04%± 0.35 | - | Passes | 56 |
| 51GLM 5Z.ai | 86.03%± 0.34 | - | Passes | no score |
| 52Kimi K2.5Moonshot AI | 85.91%± 0.34 | - | Passes | 37 |
| 53Grok 4.3xAI (SpaceXAI) | 85.84%± 0.33 | - | Rejected | no score |
| 54Gemini 3.5 Flash LiteGoogle | 85.84%± 0.34 | - | Passes | 38 |
| 55Nemotron 3 UltraNVIDIA | 85.76%± 0.34 | - | Rejected | no score |
| 56o3OpenAI | 85.59%± 0.34 | - | Passes | no score |
| 57Claude Opus 4.5 (Nonthinking)Anthropic | 85.59%± 0.37 | - | Passes | no score |
| 58Inkling SmallThinking Machines | 85.57%± 0.35 | - | Rejected | no score |
| 59Grok 4xAI (SpaceXAI) | 85.30%± 0.35 | - | Rejected | no score |
| 60Qwen 3 Max ThinkingAlibaba | 84.98%± 0.35 | - | Not yet assessed | no score |
| 61DeepSeek V3.2 (Thinking)DeepSeek | 84.92%± 0.35 | - | Passes | no score |
| 62MiMo V2.5 ProXiaomi | 84.59%± 0.45 | - | Rejected | no score |
| 63GPT 5.4 MiniOpenAI | 84.55%± 0.36 | - | Passes | 25 |
| 64Qwen 3 MaxAlibaba | 84.36%± 0.36 | - | Not yet assessed | no score |
| 65Qwen 3.8 27BAlibaba | 84.34%± 0.36 | - | Rejected | no score |
| 66MiniMax-M3MiniMax | 84.22%± 0.36 | - | Rejected | no score |
| 67Grok 4.1 Fast (Reasoning)xAI (SpaceXAI) | 84.18%± 0.36 | - | Rejected | no score |
| 68Qwen 3.5 FlashAlibaba | 84.06%± 0.36 | - | Not yet assessed | no score |
| 69Claude Sonnet 4 (Thinking)Anthropic | 83.86%± 0.36 | - | Rejected | no score |
| 70Gemini 2.5 Flash Preview (9/25) (Nonthinking)Google | 83.69%± 0.36 | - | Rejected | no score |
| 71Gemini 2.5 Flash Preview (9/25) (Thinking)Google | 83.66%± 0.36 | - | Rejected | no score |
| 72Qwen 3 Max PreviewAlibaba | 83.54%± 0.36 | - | Not yet assessed | no score |
| 73DeepSeek V3.2 (Nonthinking)DeepSeek | 83.06%± 0.38 | - | Passes | no score |
| 74MiMo V2.5Xiaomi | 82.93%± 0.37 | - | Rejected | no score |
| 75GLM 4.7Z.ai | 82.74%± 0.37 | - | Passes | no score |
| 76GPT 5 MiniOpenAI | 82.23%± 0.37 | - | Passes | no score |
| 77GLM 4.6Z.ai | 82.20%± 0.38 | - | Rejected | no score |
| 78Ling 3.0 FlashAnt Group | 82.01%± 0.38 | - | Rejected | no score |
| 79GLM 4.5Z.ai | 81.22%± 0.38 | - | Rejected | no score |
| 80Kimi K2 ThinkingMoonshot AI | 81.07%± 0.40 | - | Rejected | no score |
| 81o4 MiniOpenAI | 80.56%± 0.39 | - | Passes | no score |
| 82GPT 4.1OpenAI | 80.50%± 0.39 | - | Passes | no score |
| 83MiniMax-M2.7MiniMax | 80.43%± 0.39 | - | Rejected | no score |
| 84MiniMax-M2.5MiniMax | 80.09%± 0.39 | - | Passes | no score |
| 85Mistral Large 3Mistral AI | 79.82%± 0.47 | - | Passes | no score |
| 86Grok 4 Fast (Reasoning)xAI (SpaceXAI) | 79.70%± 0.39 | - | Rejected | no score |
| 87Claude Sonnet 4 (Nonthinking)Anthropic | 79.43%± 0.37 | - | Rejected | no score |
| 88Llama 4 MaverickMeta | 79.42%± 0.40 | - | Rejected | no score |
| 89Kimi K2 InstructMoonshot AI | 79.39%± 0.39 | - | Rejected | no score |
| 90GPT OSS 120BOpenAI | 79.17%± 0.40 | - | Passes | no score |
| 91Gemini 2.5 Flash Lite (9/25) (Thinking)Google | 79.12%± 0.40 | - | Rejected | no score |
| 92Claude Haiku 4.5 (Thinking)Anthropic | 78.72%± 0.48 | - | Passes | 32 |
| 93Gemini 2.5 Flash Lite (9/25) (Nonthinking)Google | 78.64%± 0.40 | - | Rejected | no score |
| 94GPT 4.1 MiniOpenAI | 77.22%± 0.41 | - | Passes | no score |
| 95GPT 5.4 NanoOpenAI | 77.17%± 0.43 | - | Passes | 29 |
| 96GPT 5 NanoOpenAI | 76.07%± 0.41 | - | Passes | no score |
| 97Mistral Medium 3.5Mistral AI | 75.33%± 0.42 | - | Passes | 23 |
| 98Mistral Medium 3.1 (05/2025)Mistral AI | 75.29%± 0.42 | - | Passes | no score |
| 99Grok 4.1 Fast Non-ReasoningxAI (SpaceXAI) | 75.21%± 0.42 | - | Rejected | no score |
| 100GPT 4o (2024-11-20)OpenAI | 72.56%± 0.44 | - | Passes | no score |
| 101GPT OSS 20BOpenAI | 71.64%± 0.45 | - | Passes | no score |
| 102Grok 4 Fast (Non-Reasoning)xAI (SpaceXAI) | 70.34%± 0.43 | - | Rejected | no score |
| 103Llama 4 ScoutMeta | 69.63%± 0.45 | - | Rejected | no score |
| 104Command ACohere | 69.17%± 0.46 | - | Rejected | no score |
| 105Laguna XS.2Poolside | 69.05%± 0.48 | - | Rejected | no score |
| 106Laguna M.1Poolside | 68.84%± 0.45 | - | Rejected | no score |
| 107Magistral Medium 1.2 (09/2025)Mistral AI | 68.66%± 0.43 | - | Not yet assessed | no score |
| 108Mistral Small 3.1 (03/2025)Mistral AI | 66.02%± 0.46 | - | Passes | no score |
| 109GPT 4.1 NanoOpenAI | 63.48%± 0.46 | - | Passes | no score |
| 110Magistral Small 1.2 (09/2025)Mistral AI | 62.13%± 0.45 | - | Passes | no score |
Source: Vals AI (MMLU Pro (Vals-run)) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.
Vals AI · Version 1 · Measured 1 September 2026 · Go to the source. The PrudentScore is a composition by Prudai B.V. based on publicly published figures and does not represent the view or endorsement of Vals AI, Harvey or Artificial Analysis. Figures from Artificial Analysis appear in running text only, with attribution and a link, as that party's terms require [9].
Other benchmarks
- Legal Research Bench
- Harvey LAB (Vals implementation)
- LegalBench (Vals leaderboard)
- CaseLaw v2
- TaxEval v2
- Tax Agent Bench
- CorpFin v2
- Immigration Bench
- Access to Justice
- Public Benefits Bench
- Vals Legal Index
- Vals Index
- GPQA Diamond (Vals-run)
- Artificial Analysis Intelligence Index
Claude and Anthropic, GPT and OpenAI, Gemini and Google, Grok and xAI, Kimi and Moonshot AI, Muse and Meta, Mistral, DeepSeek, GLM and Z.ai, Harvey, Vals AI and Artificial Analysis are trademarks of their respective owners. They are named for identification only and this implies no partnership, sponsorship or endorsement.