Benchmarks · Vals AI
CaseLaw v2
Measures question answering over Canadian court decisions (private dataset). Vals has archived this leaderboard (last update 4 May 2026): newer models are missing.
- Models measured
- 54
- Top
- 79.31 %Grok 4.3within the standard error of ±1.37
- Version
- 2
- Measured
- 4 May 2026
01
Ranking
At the top: Top 15
Vals AI Legal Research Bench, all-pass [1]. Hatched bars are rejected by the compliance gate.
- 1Grok 4.379.31%
- 2GPT 5.173.42%
- 3GPT 4.169.88%
- 4GPT 5 Mini68.49%
- 5Claude Opus 4.768.38%
- 6GPT 566.45%
- 7GPT 5.566.24%
- 8GPT 5.266.02%
- 9Grok 465.81%
- 10=Grok 4 Fast (Reasoning)65.70%
- 10=Kimi K2 Thinking65.70%
- 12Gemini 3.1 Pro Preview (02/26)64.84%
- 13Command A64.52%
- 14Claude Sonnet 4.663.99%
- 15Gemini 2.5 Pro63.88%
Source: Vals AI (CaseLaw v2) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.
| # Model | Score | Cost/task | Gate | PrudentScore |
|---|---|---|---|---|
| 1Grok 4.3xAI (SpaceXAI) | 79.31%± 1.37 | - | Rejected | no score |
| 2GPT 5.1OpenAI | 73.42%± 0.75 | - | Passes | no score |
| 3GPT 4.1OpenAI | 69.88%± 0.28 | - | Passes | no score |
| 4GPT 5 MiniOpenAI | 68.49%± 1.13 | - | Passes | no score |
| 5Claude Opus 4.7Anthropic | 68.38%± 0.43 | - | Passes | 54 |
| 6GPT 5OpenAI | 66.45%± 0.65 | - | Passes | no score |
| 7GPT 5.5OpenAI | 66.24%± 0.32 | - | Passes | 51 |
| 8GPT 5.2OpenAI | 66.02%± 0.57 | - | Passes | no score |
| 9Grok 4xAI (SpaceXAI) | 65.81%± 0.21 | - | Rejected | no score |
| 10=Grok 4 Fast (Reasoning)xAI (SpaceXAI) | 65.70%± 0.43 | - | Rejected | no score |
| 10=Kimi K2 ThinkingMoonshot AI | 65.70%± 1.82 | - | Rejected | no score |
| 12Gemini 3.1 Pro Preview (02/26)Google | 64.84%± 1.24 | - | Rejected | no score |
| 13Command ACohere | 64.52%± 1.09 | - | Rejected | no score |
| 14Claude Sonnet 4.6Anthropic | 63.99%± 0.56 | - | Passes | 55 |
| 15Gemini 2.5 ProGoogle | 63.88%± 0.60 | - | Not yet assessed | no score |
| 16GPT 5.4 (xhigh)OpenAI | 63.77%± 2.24 | - | Passes | 0 |
| 17Muse SparkMeta | 63.13%± 0.75 | - | Rejected | no score |
| 18Claude Opus 4.5 (Thinking)Anthropic | 62.59%± 0.11 | - | Passes | no score |
| 19Claude Sonnet 4.5 (Thinking)Anthropic | 62.16%± 1.55 | - | Passes | no score |
| 20Claude Opus 4.6 (Thinking)Anthropic | 62.06%± 0.37 | - | Passes | no score |
| 21Mistral Large 3Mistral AI | 61.41%± 0.96 | - | Passes | no score |
| 22Kimi K2.6Moonshot AI | 61.20%± 2.09 | - | Rejected | no score |
| 23MiniMax-M2.7MiniMax | 60.88%± 0.77 | - | Rejected | no score |
| 24Grok 4.1 Fast (Reasoning)xAI (SpaceXAI) | 60.45%± 0.81 | - | Rejected | no score |
| 25=GPT 4o (2024-11-20)OpenAI | 59.70%± 0.43 | - | Passes | no score |
| 25=Qwen 3.5 PlusAlibaba | 59.70%± 1.13 | - | Not yet assessed | no score |
| 27DeepSeek V4 Pro (2026-04-23)DeepSeek | 59.38%± 1.50 | - | Rejected | no score |
| 28Kimi K2.5Moonshot AI | 58.73%± 0.70 | - | Passes | 37 |
| 29Trinity Large ThinkingArcee AI | 57.88%± 1.86 | - | Rejected | no score |
| 30Claude Haiku 4.5 (Thinking)Anthropic | 56.48%± 0.92 | - | Passes | 32 |
| 31Qwen 3.5 FlashAlibaba | 55.95%± 1.16 | - | Not yet assessed | no score |
| 32=Gemini 3 Flash (12/25)Google | 55.84%± 0.11 | - | Rejected | no score |
| 32=MiniMax-M2.1MiniMax | 55.84%± 0.47 | - | Passes | no score |
| 34DeepSeek V3.2 (Thinking)DeepSeek | 55.41%± 0.47 | - | Passes | no score |
| 35=Gemini 3.1 Flash Lite PreviewGoogle | 54.98%± 0.74 | - | Not yet assessed | no score |
| 35=Qwen 3 Max ThinkingAlibaba | 54.98%± 1.49 | - | Not yet assessed | no score |
| 37GLM 4.7Z.ai | 54.88%± 0.92 | - | Passes | no score |
| 38Grok 4.20 (Reasoning)xAI (SpaceXAI) | 54.45%± 0.28 | - | Rejected | no score |
| 39DeepSeek V3.1DeepSeek | 53.91%± 0.57 | - | Passes | no score |
| 40MiniMax-M2.5MiniMax | 53.48%± 0.47 | - | Passes | no score |
| 41Qwen 3.6 27BAlibaba | 53.16%± 2.17 | - | Rejected | no score |
| 42Gemini 3 Pro (11/25)Google | 53.05%± 0.19 | - | Rejected | no score |
| 43=Gemma 4 31B ITGoogle | 52.63%± 1.34 | - | Passes | no score |
| 43=GPT 5 NanoOpenAI | 52.63%± 1.02 | - | Passes | no score |
| 45GLM 5Z.ai | 52.52%± 1.73 | - | Passes | no score |
| 46GPT 5.4 NanoOpenAI | 51.88%± 1.09 | - | Passes | 29 |
| 47GPT 5.4 MiniOpenAI | 51.66%± 0.93 | - | Passes | 25 |
| 48GLM 5.1Z.ai | 51.55%± 1.13 | - | Rejected | no score |
| 49Qwen 3.6 PlusAlibaba | 51.45%± 1.16 | - | Not yet assessed | no score |
| 50GPT OSS 120BOpenAI | 48.77%± 0.88 | - | Passes | no score |
| 51Qwen 3.6 Max PreviewAlibaba | 47.91%± 1.11 | - | Not yet assessed | no score |
| 52Qwen 3 MaxAlibaba | 47.48%± 0.47 | - | Not yet assessed | no score |
| 53Mistral Medium 3.5Mistral AI | 44.16%± 1.06 | - | Passes | 23 |
| 54GPT OSS 20BOpenAI | 43.84%± 0.94 | - | Passes | no score |
Source: Vals AI (CaseLaw v2) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.
Vals AI · Version 2 · Measured 4 May 2026 · Go to the source. The PrudentScore is a composition by Prudai B.V. based on publicly published figures and does not represent the view or endorsement of Vals AI, Harvey or Artificial Analysis. Figures from Artificial Analysis appear in running text only, with attribution and a link, as that party's terms require [9].
Other benchmarks
- Legal Research Bench
- Harvey LAB (Vals implementation)
- LegalBench (Vals leaderboard)
- TaxEval v2
- Tax Agent Bench
- CorpFin v2
- Immigration Bench
- Access to Justice
- Public Benefits Bench
- Vals Legal Index
- Vals Index
- GPQA Diamond (Vals-run)
- MMLU Pro (Vals-run)
- Artificial Analysis Intelligence Index
Claude and Anthropic, GPT and OpenAI, Gemini and Google, Grok and xAI, Kimi and Moonshot AI, Muse and Meta, Mistral, DeepSeek, GLM and Z.ai, Harvey, Vals AI and Artificial Analysis are trademarks of their respective owners. They are named for identification only and this implies no partnership, sponsorship or endorsement.