Skip to content

Benchmarks · Vals AI

MMLU Pro (Vals-run)

General knowledge benchmark: multiple-choice questions across 14 subjects, including law. Vals marks the leaderboard "Archived Benchmark": "Since performance on this benchmark has saturated, we no longer run this benchmark on new model releases" (last update 1 September 2026).

Models measured
110
Top
92.38 %Claude Fable 5.1within the standard error of ±0.27
Version
1
Measured
1 September 2026
Go to the sourceThis benchmark does not count towards the PrudentScore.

01

Ranking

At the top: Top 15

Vals AI Legal Research Bench, all-pass [1]. Hatched bars are rejected by the compliance gate.

  1. 1Claude Fable 5.192.38%
  2. 2Claude Opus 591.59%
  3. 3Claude Fable 591.50%
  4. 4Gemini 3.1 Pro Preview (02/26)90.99%
  5. 5Gemini 3.8 Flash90.22%
  6. 6Gemini 3.7 Flash90.12%
  7. 7Gemini 3 Pro (11/25)90.10%
  8. 8Claude Opus 4.789.87%
  9. 9Claude Opus 4.889.58%
  10. 10Gemini 3.5 Flash89.52%
  11. 11Grok 4.689.40%
  12. 12Qwen 3.7 Max89.31%
  13. 13Gemini 3.6 Flash89.28%
  14. 14Grok 4.589.22%
  15. 15Claude Opus 4.6 (Thinking)89.11%

Source: Vals AI (MMLU Pro (Vals-run)) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.

Passes the compliance gateRejected by the compliance gate
MMLU Pro (Vals-run) 1
# ModelScoreCost/taskGatePrudentScore
1Claude Fable 5.1Anthropic92.38%± 0.27-Rejectedno score
2Claude Opus 5Anthropic91.59%± 0.28-Passes72
3Claude Fable 5Anthropic91.50%± 0.28-Rejectedno score
4Gemini 3.1 Pro Preview (02/26)Google90.99%± 0.28-Rejectedno score
5Gemini 3.8 FlashGoogle90.22%± 0.29-Passes62
6Gemini 3.7 FlashGoogle90.12%± 0.30-Passes58
7Gemini 3 Pro (11/25)Google90.10%± 0.29-Rejectedno score
8Claude Opus 4.7Anthropic89.87%± 0.30-Passes54
9Claude Opus 4.8Anthropic89.58%± 0.30-Passes62
10Gemini 3.5 FlashGoogle89.52%± 0.31-Passes47
11Grok 4.6xAI (SpaceXAI)89.40%± 0.30-Rejectedno score
12Qwen 3.7 MaxAlibaba89.31%± 0.30-Not yet assessedno score
13Gemini 3.6 FlashGoogle89.28%± 0.30-Passes46
14Grok 4.5xAI (SpaceXAI)89.22%± 0.31-Rejectedno score
15Claude Opus 4.6 (Thinking)Anthropic89.11%± 0.46-Passesno score
16GPT-5.6 SolOpenAI89.10%± 0.31-Passes57
17Muse Spark 1.1Meta88.73%± 0.32-Rejectedno score
18Qwen 3.8 MaxAlibaba88.60%± 0.31-Not yet assessedno score
19Gemini 3 Flash (12/25)Google88.59%± 0.32-Rejectedno score
20Muse Spark 1.2Meta88.28%± 0.32-Rejectedno score
21GPT 5.5OpenAI88.14%± 0.32-Passes51
22Kimi K3Moonshot AI87.97%± 0.32-Rejectedno score
23Claude Opus 4.1 (Thinking)Anthropic87.92%± 0.33-Rejectedno score
24Qwen 3.6 PlusAlibaba87.67%± 0.32-Not yet assessedno score
25Kimi K2.6Moonshot AI87.57%± 0.33-Rejectedno score
26Claude Sonnet 5Anthropic87.55%± 0.37-Passes59
27GPT 5.4 (xhigh)OpenAI87.48%± 0.42-Passes0
28Claude Sonnet 4.5 (Thinking)Anthropic87.36%± 0.39-Passesno score
29Claude Sonnet 4.6Anthropic87.34%± 0.43-Passes55
30Muse SparkMeta87.32%± 0.33-Rejectedno score
31Claude Opus 4.5 (Thinking)Anthropic87.26%± 0.38-Passesno score
32DeepSeek V4 Pro (2026-04-23)DeepSeek87.25%± 0.34-Rejectedno score
33Claude Opus 4.1 (Nonthinking)Anthropic87.21%± 0.33-Rejectedno score
34Qwen 3.5 PlusAlibaba87.18%± 0.33-Not yet assessedno score
35MiniMax-M2.1MiniMax87.05%± 0.33-Passesno score
36DeepSeek V4 Pro (0813)DeepSeek86.97%± 0.34-Rejectedno score
37GLM 5.1Z.ai86.90%± 0.33-Rejectedno score
38GLM 5.3Z.ai86.77%± 0.34-Rejectedno score
39GLM 5.2Z.ai86.71%± 0.33-Rejectedno score
40GPT-5.6 TerraOpenAI86.66%± 0.33-Passes53
41GPT 5OpenAI86.54%± 0.34-Passesno score
42GPT 5.1OpenAI86.38%± 0.34-Passesno score
43InklingThinking Machines86.30%± 0.34-Rejectedno score
44Grok 4.20 (Reasoning)xAI (SpaceXAI)86.25%± 0.34-Rejectedno score
45Gemini 3.1 Flash Lite PreviewGoogle86.24%± 0.34-Not yet assessedno score
46GPT 5.2OpenAI86.23%± 0.34-Passesno score
47DeepSeek V4 Flash 0731DeepSeek86.21%± 0.34-Rejectedno score
48Claude Opus 4 (Nonthinking)Anthropic86.17%± 0.34-Rejectedno score
49GLM 5.3 FlashZ.ai86.06%± 0.34-Rejectedno score
50GPT-5.6 LunaOpenAI86.04%± 0.35-Passes56
51GLM 5Z.ai86.03%± 0.34-Passesno score
52Kimi K2.5Moonshot AI85.91%± 0.34-Passes37
53Grok 4.3xAI (SpaceXAI)85.84%± 0.33-Rejectedno score
54Gemini 3.5 Flash LiteGoogle85.84%± 0.34-Passes38
55Nemotron 3 UltraNVIDIA85.76%± 0.34-Rejectedno score
56o3OpenAI85.59%± 0.34-Passesno score
57Claude Opus 4.5 (Nonthinking)Anthropic85.59%± 0.37-Passesno score
58Inkling SmallThinking Machines85.57%± 0.35-Rejectedno score
59Grok 4xAI (SpaceXAI)85.30%± 0.35-Rejectedno score
60Qwen 3 Max ThinkingAlibaba84.98%± 0.35-Not yet assessedno score
61DeepSeek V3.2 (Thinking)DeepSeek84.92%± 0.35-Passesno score
62MiMo V2.5 ProXiaomi84.59%± 0.45-Rejectedno score
63GPT 5.4 MiniOpenAI84.55%± 0.36-Passes25
64Qwen 3 MaxAlibaba84.36%± 0.36-Not yet assessedno score
65Qwen 3.8 27BAlibaba84.34%± 0.36-Rejectedno score
66MiniMax-M3MiniMax84.22%± 0.36-Rejectedno score
67Grok 4.1 Fast (Reasoning)xAI (SpaceXAI)84.18%± 0.36-Rejectedno score
68Qwen 3.5 FlashAlibaba84.06%± 0.36-Not yet assessedno score
69Claude Sonnet 4 (Thinking)Anthropic83.86%± 0.36-Rejectedno score
70Gemini 2.5 Flash Preview (9/25) (Nonthinking)Google83.69%± 0.36-Rejectedno score
71Gemini 2.5 Flash Preview (9/25) (Thinking)Google83.66%± 0.36-Rejectedno score
72Qwen 3 Max PreviewAlibaba83.54%± 0.36-Not yet assessedno score
73DeepSeek V3.2 (Nonthinking)DeepSeek83.06%± 0.38-Passesno score
74MiMo V2.5Xiaomi82.93%± 0.37-Rejectedno score
75GLM 4.7Z.ai82.74%± 0.37-Passesno score
76GPT 5 MiniOpenAI82.23%± 0.37-Passesno score
77GLM 4.6Z.ai82.20%± 0.38-Rejectedno score
78Ling 3.0 FlashAnt Group82.01%± 0.38-Rejectedno score
79GLM 4.5Z.ai81.22%± 0.38-Rejectedno score
80Kimi K2 ThinkingMoonshot AI81.07%± 0.40-Rejectedno score
81o4 MiniOpenAI80.56%± 0.39-Passesno score
82GPT 4.1OpenAI80.50%± 0.39-Passesno score
83MiniMax-M2.7MiniMax80.43%± 0.39-Rejectedno score
84MiniMax-M2.5MiniMax80.09%± 0.39-Passesno score
85Mistral Large 3Mistral AI79.82%± 0.47-Passesno score
86Grok 4 Fast (Reasoning)xAI (SpaceXAI)79.70%± 0.39-Rejectedno score
87Claude Sonnet 4 (Nonthinking)Anthropic79.43%± 0.37-Rejectedno score
88Llama 4 MaverickMeta79.42%± 0.40-Rejectedno score
89Kimi K2 InstructMoonshot AI79.39%± 0.39-Rejectedno score
90GPT OSS 120BOpenAI79.17%± 0.40-Passesno score
91Gemini 2.5 Flash Lite (9/25) (Thinking)Google79.12%± 0.40-Rejectedno score
92Claude Haiku 4.5 (Thinking)Anthropic78.72%± 0.48-Passes32
93Gemini 2.5 Flash Lite (9/25) (Nonthinking)Google78.64%± 0.40-Rejectedno score
94GPT 4.1 MiniOpenAI77.22%± 0.41-Passesno score
95GPT 5.4 NanoOpenAI77.17%± 0.43-Passes29
96GPT 5 NanoOpenAI76.07%± 0.41-Passesno score
97Mistral Medium 3.5Mistral AI75.33%± 0.42-Passes23
98Mistral Medium 3.1 (05/2025)Mistral AI75.29%± 0.42-Passesno score
99Grok 4.1 Fast Non-ReasoningxAI (SpaceXAI)75.21%± 0.42-Rejectedno score
100GPT 4o (2024-11-20)OpenAI72.56%± 0.44-Passesno score
101GPT OSS 20BOpenAI71.64%± 0.45-Passesno score
102Grok 4 Fast (Non-Reasoning)xAI (SpaceXAI)70.34%± 0.43-Rejectedno score
103Llama 4 ScoutMeta69.63%± 0.45-Rejectedno score
104Command ACohere69.17%± 0.46-Rejectedno score
105Laguna XS.2Poolside69.05%± 0.48-Rejectedno score
106Laguna M.1Poolside68.84%± 0.45-Rejectedno score
107Magistral Medium 1.2 (09/2025)Mistral AI68.66%± 0.43-Not yet assessedno score
108Mistral Small 3.1 (03/2025)Mistral AI66.02%± 0.46-Passesno score
109GPT 4.1 NanoOpenAI63.48%± 0.46-Passesno score
110Magistral Small 1.2 (09/2025)Mistral AI62.13%± 0.45-Passesno score

Source: Vals AI (MMLU Pro (Vals-run)) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.

Vals AI · Version 1 · Measured 1 September 2026 · Go to the source. The PrudentScore is a composition by Prudai B.V. based on publicly published figures and does not represent the view or endorsement of Vals AI, Harvey or Artificial Analysis. Figures from Artificial Analysis appear in running text only, with attribution and a link, as that party's terms require [9].

Other benchmarks

Claude and Anthropic, GPT and OpenAI, Gemini and Google, Grok and xAI, Kimi and Moonshot AI, Muse and Meta, Mistral, DeepSeek, GLM and Z.ai, Harvey, Vals AI and Artificial Analysis are trademarks of their respective owners. They are named for identification only and this implies no partnership, sponsorship or endorsement.