Skip to content

Benchmarks · Vals AI

CaseLaw v2

Measures question answering over Canadian court decisions (private dataset). Vals has archived this leaderboard (last update 4 May 2026): newer models are missing.

Models measured
54
Top
79.31 %Grok 4.3within the standard error of ±1.37
Version
2
Measured
4 May 2026
Go to the sourceThis benchmark does not count towards the PrudentScore.

01

Ranking

At the top: Top 15

Vals AI Legal Research Bench, all-pass [1]. Hatched bars are rejected by the compliance gate.

  1. 1Grok 4.379.31%
  2. 2GPT 5.173.42%
  3. 3GPT 4.169.88%
  4. 4GPT 5 Mini68.49%
  5. 5Claude Opus 4.768.38%
  6. 6GPT 566.45%
  7. 7GPT 5.566.24%
  8. 8GPT 5.266.02%
  9. 9Grok 465.81%
  10. 10=Grok 4 Fast (Reasoning)65.70%
  11. 10=Kimi K2 Thinking65.70%
  12. 12Gemini 3.1 Pro Preview (02/26)64.84%
  13. 13Command A64.52%
  14. 14Claude Sonnet 4.663.99%
  15. 15Gemini 2.5 Pro63.88%

Source: Vals AI (CaseLaw v2) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.

Passes the compliance gateRejected by the compliance gate
CaseLaw v2 2
# ModelScoreCost/taskGatePrudentScore
1Grok 4.3xAI (SpaceXAI)79.31%± 1.37-Rejectedno score
2GPT 5.1OpenAI73.42%± 0.75-Passesno score
3GPT 4.1OpenAI69.88%± 0.28-Passesno score
4GPT 5 MiniOpenAI68.49%± 1.13-Passesno score
5Claude Opus 4.7Anthropic68.38%± 0.43-Passes54
6GPT 5OpenAI66.45%± 0.65-Passesno score
7GPT 5.5OpenAI66.24%± 0.32-Passes51
8GPT 5.2OpenAI66.02%± 0.57-Passesno score
9Grok 4xAI (SpaceXAI)65.81%± 0.21-Rejectedno score
10=Grok 4 Fast (Reasoning)xAI (SpaceXAI)65.70%± 0.43-Rejectedno score
10=Kimi K2 ThinkingMoonshot AI65.70%± 1.82-Rejectedno score
12Gemini 3.1 Pro Preview (02/26)Google64.84%± 1.24-Rejectedno score
13Command ACohere64.52%± 1.09-Rejectedno score
14Claude Sonnet 4.6Anthropic63.99%± 0.56-Passes55
15Gemini 2.5 ProGoogle63.88%± 0.60-Not yet assessedno score
16GPT 5.4 (xhigh)OpenAI63.77%± 2.24-Passes0
17Muse SparkMeta63.13%± 0.75-Rejectedno score
18Claude Opus 4.5 (Thinking)Anthropic62.59%± 0.11-Passesno score
19Claude Sonnet 4.5 (Thinking)Anthropic62.16%± 1.55-Passesno score
20Claude Opus 4.6 (Thinking)Anthropic62.06%± 0.37-Passesno score
21Mistral Large 3Mistral AI61.41%± 0.96-Passesno score
22Kimi K2.6Moonshot AI61.20%± 2.09-Rejectedno score
23MiniMax-M2.7MiniMax60.88%± 0.77-Rejectedno score
24Grok 4.1 Fast (Reasoning)xAI (SpaceXAI)60.45%± 0.81-Rejectedno score
25=GPT 4o (2024-11-20)OpenAI59.70%± 0.43-Passesno score
25=Qwen 3.5 PlusAlibaba59.70%± 1.13-Not yet assessedno score
27DeepSeek V4 Pro (2026-04-23)DeepSeek59.38%± 1.50-Rejectedno score
28Kimi K2.5Moonshot AI58.73%± 0.70-Passes37
29Trinity Large ThinkingArcee AI57.88%± 1.86-Rejectedno score
30Claude Haiku 4.5 (Thinking)Anthropic56.48%± 0.92-Passes32
31Qwen 3.5 FlashAlibaba55.95%± 1.16-Not yet assessedno score
32=Gemini 3 Flash (12/25)Google55.84%± 0.11-Rejectedno score
32=MiniMax-M2.1MiniMax55.84%± 0.47-Passesno score
34DeepSeek V3.2 (Thinking)DeepSeek55.41%± 0.47-Passesno score
35=Gemini 3.1 Flash Lite PreviewGoogle54.98%± 0.74-Not yet assessedno score
35=Qwen 3 Max ThinkingAlibaba54.98%± 1.49-Not yet assessedno score
37GLM 4.7Z.ai54.88%± 0.92-Passesno score
38Grok 4.20 (Reasoning)xAI (SpaceXAI)54.45%± 0.28-Rejectedno score
39DeepSeek V3.1DeepSeek53.91%± 0.57-Passesno score
40MiniMax-M2.5MiniMax53.48%± 0.47-Passesno score
41Qwen 3.6 27BAlibaba53.16%± 2.17-Rejectedno score
42Gemini 3 Pro (11/25)Google53.05%± 0.19-Rejectedno score
43=Gemma 4 31B ITGoogle52.63%± 1.34-Passesno score
43=GPT 5 NanoOpenAI52.63%± 1.02-Passesno score
45GLM 5Z.ai52.52%± 1.73-Passesno score
46GPT 5.4 NanoOpenAI51.88%± 1.09-Passes29
47GPT 5.4 MiniOpenAI51.66%± 0.93-Passes25
48GLM 5.1Z.ai51.55%± 1.13-Rejectedno score
49Qwen 3.6 PlusAlibaba51.45%± 1.16-Not yet assessedno score
50GPT OSS 120BOpenAI48.77%± 0.88-Passesno score
51Qwen 3.6 Max PreviewAlibaba47.91%± 1.11-Not yet assessedno score
52Qwen 3 MaxAlibaba47.48%± 0.47-Not yet assessedno score
53Mistral Medium 3.5Mistral AI44.16%± 1.06-Passes23
54GPT OSS 20BOpenAI43.84%± 0.94-Passesno score

Source: Vals AI (CaseLaw v2) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.

Vals AI · Version 2 · Measured 4 May 2026 · Go to the source. The PrudentScore is a composition by Prudai B.V. based on publicly published figures and does not represent the view or endorsement of Vals AI, Harvey or Artificial Analysis. Figures from Artificial Analysis appear in running text only, with attribution and a link, as that party's terms require [9].

Other benchmarks

Claude and Anthropic, GPT and OpenAI, Gemini and Google, Grok and xAI, Kimi and Moonshot AI, Muse and Meta, Mistral, DeepSeek, GLM and Z.ai, Harvey, Vals AI and Artificial Analysis are trademarks of their respective owners. They are named for identification only and this implies no partnership, sponsorship or endorsement.