Benchmarks · Vals AI
Legal Research Bench
Note: the questions cover US law; Vals has no benchmark for Dutch or EU law. Measures whether an AI agent with search tools fully answers a legal research question correctly: 208 test questions (private test set, from 413 expert-authored, peer-reviewed questions) across eight practice areas, graded by GPT-5.4 as judge against a rubric; "all-pass" counts only when every criterion is met.
- Models measured
- 91
- Top
- 55.77 %Gemini 4 Carbonwithin the standard error of ±3.45
- Version
- 1
- Measured
- 9 November 2026
01
Ranking
At the top: Top 15
Vals AI Legal Research Bench, all-pass [1]. Hatched bars are rejected by the compliance gate.
- 1Gemini 4 Carbon55.77%
- 2=Claude Fable 5.155.29%
- 2=Claude Opus 555.29%
- 2=Muse Spark 1.3 (Max)55.29%
- 5Gemini 4 Argon54.81%
- 6Wonder Full53.37%
- 7=Claude Opus 5.550.48%
- 7=Wonder50.48%
- 9Claude Fable 549.52%
- 10=GLM 5.349.04%
- 10=Muse Ember49.04%
- 12=Claude Sonnet 5.548.08%
- 12=GPT-5.6 Sol48.08%
- 12=Grok 4.648.08%
- 15Qwen 3.8 Max47.60%
Source: Vals AI (Legal Research Bench) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.
Hover a dot for the figures. Models without a published cost per task are not in this chart.
Cost per completed task against the legal-research score: Gemini 4 Argon: $6.45, 54.81%, Passes the compliance gate, on the Pareto frontier. Gemini 4 Carbon: $8.74, 55.77%, Passes the compliance gate, on the Pareto frontier. Claude Opus 5: $6.76, 55.29%, Passes the compliance gate, on the Pareto frontier. Claude Opus 5.5: $23.61, 50.48%, Passes the compliance gate. Claude Opus 4.8: $2.82, 43.75%, Passes the compliance gate. Gemini 3.8 Flash: $1.63, 38.94%, Passes the compliance gate, on the Pareto frontier. Claude Sonnet 5.5: $13.66, 48.08%, Passes the compliance gate. Claude Haiku 5.5: $1.65, 43.75%, Passes the compliance gate, on the Pareto frontier. Claude Sonnet 5: $2.72, 41.83%, Passes the compliance gate. GPT-6.1 Sol: $3.38, 38.46%, Passes the compliance gate. Gemini 3.7 Flash: $0.76, 34.62%, Passes the compliance gate, on the Pareto frontier. GPT-5.6 Sol: $21.61, 48.08%, Passes the compliance gate. GPT-5.6 Luna: $0.84, 36.54%, Passes the compliance gate, on the Pareto frontier. GPT-6 Astra: $10.58, 39.42%, Passes the compliance gate. Claude Sonnet 4.6: $2.30, 38.46%, Passes the compliance gate. Claude Opus 4.7: $3.06, 38.46%, Passes the compliance gate. GPT-6 Luna: $0.44, 30.29%, Passes the compliance gate, on the Pareto frontier. GPT-5.6 Terra: $7.91, 41.35%, Passes the compliance gate. GPT 5.5: $7.40, 40.38%, Passes the compliance gate. Gemini 3.5 Flash: $1.31, 30.77%, Passes the compliance gate. Gemini 3.6 Flash: $0.47, 25.00%, Passes the compliance gate. GPT-6 Sol: $4.90, 28.85%, Passes the compliance gate. GPT-6.1 Terra: $3.22, 25.96%, Passes the compliance gate. Gemini 3.5 Flash Lite: $0.12, 13.94%, Passes the compliance gate, on the Pareto frontier. Kimi K2.5: $0.23, 15.87%, Passes the compliance gate, on the Pareto frontier. Claude Haiku 4.5 (Thinking): $0.27, 10.58%, Passes the compliance gate. GPT 5.4 Nano: $0.14, 6.25%, Passes the compliance gate. GPT 5.4 Mini: $1.86, 12.50%, Passes the compliance gate. Mistral Medium 3.5: $1.31, 9.13%, Passes the compliance gate. Claude Fable 5.1: $23.06, 55.29%, Rejected by the compliance gate. Muse Spark 1.3 (Max): $0.59, 55.29%, Rejected by the compliance gate. Wonder Full: $3.60, 53.37%, Rejected by the compliance gate. Wonder: $3.98, 50.48%, Rejected by the compliance gate. Claude Fable 5: $9.79, 49.52%, Rejected by the compliance gate. GLM 5.3: $2.24, 49.04%, Rejected by the compliance gate. Muse Ember: $0.06, 49.04%, Rejected by the compliance gate. Grok 4.6: $1.53, 48.08%, Rejected by the compliance gate. Qwen 3.8 Max: $2.49, 47.60%, Gate not yet assessed. Grok 4.7: $4.92, 47.12%, Rejected by the compliance gate. MiMo V2.6 Pro: $0.18, 47.12%, Rejected by the compliance gate. Inkling 2 Small: $0.29, 46.63%, Rejected by the compliance gate. Claude Fable 5.2: $3.96, 46.15%, Rejected by the compliance gate. Kimi K3: $3.47, 46.15%, Rejected by the compliance gate. DeepSeek V4.1 Flash 1017: $0.70, 45.19%, Rejected by the compliance gate. GLM 5.3 Flash: $0.24, 45.19%, Rejected by the compliance gate. Hy4 Preview: $0.85, 45.19%, Rejected by the compliance gate. Ember-1: $3.67, 44.71%, Gate not yet assessed. Muse Spark 1.2: $0.51, 43.75%, Rejected by the compliance gate. GLM Legal: $0.05, 41.83%, Rejected by the compliance gate. DeepSeek V4.1 Flash: $0.25, 41.35%, Rejected by the compliance gate. Hy4: $2.22, 41.35%, Rejected by the compliance gate. DeepSeek V4 Pro (0813): $1.09, 40.87%, Rejected by the compliance gate. Muse Spark 1.3: $0.65, 40.87%, Rejected by the compliance gate. Step 5 Preview: $0.84, 40.38%, Rejected by the compliance gate. Muse Spark Open: $1.56, 38.46%, Rejected by the compliance gate. Grok 4.5: $1.09, 37.98%, Rejected by the compliance gate. MiMo V2.6 Flash: $0.07, 37.98%, Rejected by the compliance gate. Muse Spark 1.1: $0.38, 37.98%, Rejected by the compliance gate. Kimi K3.1 Pro: $7.23, 37.50%, Rejected by the compliance gate. Qwen 3.8 27B: $2.16, 36.06%, Rejected by the compliance gate. Mistral Large 4 (GA, 26.10): $1.09, 35.58%, Gate not yet assessed. MiniMax-M3.1-Flash: $0.44, 34.13%, Rejected by the compliance gate. Mistral Large 4 (preview): $2.72, 31.73%, Rejected by the compliance gate. GLM 5.2: $0.89, 31.25%, Rejected by the compliance gate. DeepSeek V4 Flash 0731: $0.28, 30.29%, Rejected by the compliance gate. MiniMax-M3: $0.34, 29.81%, Rejected by the compliance gate. Mercury 3: $0.07, 28.85%, Gate not yet assessed. Inkling: $0.49, 28.36%, Rejected by the compliance gate. GLM 5.1: $0.39, 27.89%, Rejected by the compliance gate. Inkling Small: $0.14, 25.48%, Rejected by the compliance gate. Qwen 3.7 Max: $1.40, 25.48%, Gate not yet assessed. DeepSeek V4 Pro (2026-04-23): $0.68, 23.08%, Rejected by the compliance gate. Gemini 3.1 Pro Preview (02/26): $1.10, 20.67%, Rejected by the compliance gate. Gemini 3 Flash (12/25): $0.14, 18.27%, Rejected by the compliance gate. Qwen 3.7 Plus: $0.30, 16.35%, Gate not yet assessed. Kimi K2.6: $1.52, 15.87%, Rejected by the compliance gate. MiMo V2.5 Pro: $0.08, 15.87%, Rejected by the compliance gate. Grok 4.3: $0.20, 15.38%, Rejected by the compliance gate. Qwen 3.6 Plus: $0.70, 14.90%, Gate not yet assessed. Grok 4.20 (Reasoning): $0.26, 13.94%, Rejected by the compliance gate. MiniMax-M2.7: $0.22, 10.58%, Rejected by the compliance gate. Nemotron 3.5 Super VL: $0.06, 10.10%, Rejected by the compliance gate. MiMo V2.5: $0.03, 9.13%, Rejected by the compliance gate. Mercury 2.5: $0.06, 4.33%, Gate not yet assessed. Gemini 3.1 Flash Lite Preview: $0.01, 3.37%, Gate not yet assessed. Nemotron 3.5 Lightning: < $0.01, 2.88%, Rejected by the compliance gate. Laguna M.1: $0.22, 2.40%, Rejected by the compliance gate. Laguna XS.2: $0.11, 0.96%, Rejected by the compliance gate. Ling 3.0 Flash: $0.04, 0.00%, Rejected by the compliance gate.
| # Model | Score | Cost/task | Gate | PrudentScore |
|---|---|---|---|---|
| 1Gemini 4 CarbonGoogle | 55.77%± 3.45 | $8.74 | Passes | 77 |
| 2=Claude Fable 5.1Anthropic | 55.29%± 3.46 | $23.06 | Rejected | no score |
| 2=Claude Opus 5Anthropic | 55.29%± 3.46 | $6.76 | Passes | 72 |
| 2=Muse Spark 1.3 (Max)Meta | 55.29%± 3.46 | $0.59 | Rejected | no score |
| 5Gemini 4 ArgonGoogle | 54.81%± 3.46 | $6.45 | Passes | 79 |
| 6Wonder FullMeta | 53.37%± 3.47 | $3.60 | Rejected | no score |
| 7=Claude Opus 5.5Anthropic | 50.48%± 3.48 | $23.61 | Passes | 63 |
| 7=WonderMeta | 50.48%± 3.48 | $3.98 | Rejected | no score |
| 9Claude Fable 5Anthropic | 49.52%± 3.48 | $9.79 | Rejected | no score |
| 10=GLM 5.3Z.ai | 49.04%± 3.48 | $2.24 | Rejected | no score |
| 10=Muse EmberMeta | 49.04%± 3.48 | $0.06 | Rejected | no score |
| 12=Claude Sonnet 5.5Anthropic | 48.08%± 3.47 | $13.66 | Passes | 62 |
| 12=GPT-5.6 SolOpenAI | 48.08%± 3.47 | $21.61 | Passes | 57 |
| 12=Grok 4.6xAI (SpaceXAI) | 48.08%± 3.47 | $1.53 | Rejected | no score |
| 15Qwen 3.8 MaxAlibaba | 47.60%± 3.47 | $2.49 | Not yet assessed | no score |
| 16=Grok 4.7xAI (SpaceXAI) | 47.12%± 3.47 | $4.92 | Rejected | no score |
| 16=MiMo V2.6 ProXiaomi | 47.12%± 3.47 | $0.18 | Rejected | no score |
| 18Inkling 2 SmallThinking Machines | 46.63%± 3.47 | $0.29 | Rejected | no score |
| 19=Claude Fable 5.2Anthropic | 46.15%± 3.46 | $3.96 | Rejected | no score |
| 19=Kimi K3Moonshot AI | 46.15%± 3.46 | $3.47 | Rejected | no score |
| 21=DeepSeek V4.1 Flash 1017DeepSeek | 45.19%± 3.46 | $0.70 | Rejected | no score |
| 21=GLM 5.3 FlashZ.ai | 45.19%± 3.46 | $0.24 | Rejected | no score |
| 21=Hy4 PreviewTencent | 45.19%± 3.46 | $0.85 | Rejected | no score |
| 24Ember-1Fireworks AI | 44.71%± 3.46 | $3.67 | Not yet assessed | no score |
| 25=Claude Haiku 5.5Anthropic | 43.75%± 3.45 | $1.65 | Passes | 59 |
| 25=Claude Opus 4.8Anthropic | 43.75%± 3.45 | $2.82 | Passes | 62 |
| 25=Muse Spark 1.2Meta | 43.75%± 3.45 | $0.51 | Rejected | no score |
| 28=Claude Sonnet 5Anthropic | 41.83%± 3.43 | $2.72 | Passes | 59 |
| 28=GLM LegalZ.ai | 41.83%± 3.43 | $0.05 | Rejected | no score |
| 30=DeepSeek V4.1 FlashDeepSeek | 41.35%± 3.42 | $0.25 | Rejected | no score |
| 30=GPT-5.6 TerraOpenAI | 41.35%± 3.42 | $7.91 | Passes | 53 |
| 30=Hy4Tencent | 41.35%± 3.42 | $2.22 | Rejected | no score |
| 33=DeepSeek V4 Pro (0813)DeepSeek | 40.87%± 3.42 | $1.09 | Rejected | no score |
| 33=Muse Spark 1.3Meta | 40.87%± 3.42 | $0.65 | Rejected | no score |
| 35=GPT 5.5OpenAI | 40.38%± 3.41 | $7.40 | Passes | 51 |
| 35=Step 5 PreviewStepFun | 40.38%± 3.41 | $0.84 | Rejected | no score |
| 37GPT-6 AstraOpenAI | 39.42%± 3.40 | $10.58 | Passes | 56 |
| 38Gemini 3.8 FlashGoogle | 38.94%± 3.39 | $1.63 | Passes | 62 |
| 39=Claude Opus 4.7Anthropic | 38.46%± 3.38 | $3.06 | Passes | 54 |
| 39=Claude Sonnet 4.6Anthropic | 38.46%± 3.38 | $2.30 | Passes | 55 |
| 39=GPT-6.1 SolOpenAI | 38.46%± 3.38 | $3.38 | Passes | 59 |
| 39=Muse Spark OpenMeta | 38.46%± 3.38 | $1.56 | Rejected | no score |
| 43=Grok 4.5xAI (SpaceXAI) | 37.98%± 3.37 | $1.09 | Rejected | no score |
| 43=MiMo V2.6 FlashXiaomi | 37.98%± 3.37 | $0.07 | Rejected | no score |
| 43=Muse Spark 1.1Meta | 37.98%± 3.37 | $0.38 | Rejected | no score |
| 46Kimi K3.1 ProMoonshot AI | 37.50%± 3.37 | $7.23 | Rejected | no score |
| 47GPT-5.6 LunaOpenAI | 36.54%± 3.35 | $0.84 | Passes | 56 |
| 48Qwen 3.8 27BAlibaba | 36.06%± 3.34 | $2.16 | Rejected | no score |
| 49Mistral Large 4 (GA, 26.10)Mistral AI | 35.58%± 3.33 | $1.09 | Not yet assessed | no score |
| 50Gemini 3.7 FlashGoogle | 34.62%± 3.31 | $0.76 | Passes | 58 |
| 51MiniMax-M3.1-FlashMiniMax | 34.13%± 3.30 | $0.44 | Rejected | no score |
| 52Mistral Large 4 (preview)Mistral AI | 31.73%± 3.23 | $2.72 | Rejected | no score |
| 53GLM 5.2Z.ai | 31.25%± 3.22 | $0.89 | Rejected | no score |
| 54Gemini 3.5 FlashGoogle | 30.77%± 3.21 | $1.31 | Passes | 47 |
| 55=DeepSeek V4 Flash 0731DeepSeek | 30.29%± 3.19 | $0.28 | Rejected | no score |
| 55=GPT-6 LunaOpenAI | 30.29%± 3.19 | $0.44 | Passes | 54 |
| 57MiniMax-M3MiniMax | 29.81%± 3.18 | $0.34 | Rejected | no score |
| 58=Beam 1.0Reflection AI | 28.85%± 3.15 | - | Rejected | no score |
| 58=GPT-6 SolOpenAI | 28.85%± 3.15 | $4.90 | Passes | 45 |
| 58=Mercury 3Inception Labs | 28.85%± 3.15 | $0.07 | Not yet assessed | no score |
| 61InklingThinking Machines | 28.36%± 3.13 | $0.49 | Rejected | no score |
| 62GLM 5.1Z.ai | 27.89%± 3.12 | $0.39 | Rejected | no score |
| 63GPT-6.1 TerraOpenAI | 25.96%± 3.05 | $3.22 | Passes | 44 |
| 64=Inkling SmallThinking Machines | 25.48%± 3.03 | $0.14 | Rejected | no score |
| 64=Qwen 3.7 MaxAlibaba | 25.48%± 3.03 | $1.40 | Not yet assessed | no score |
| 66Gemini 3.6 FlashGoogle | 25.00%± 3.01 | $0.47 | Passes | 46 |
| 67DeepSeek V4 Pro (2026-04-23)DeepSeek | 23.08%± 2.93 | $0.68 | Rejected | no score |
| 68Gemini 3.1 Pro Preview (02/26)Google | 20.67%± 2.81 | $1.10 | Rejected | no score |
| 69Gemini 3 Flash (12/25)Google | 18.27%± 2.69 | $0.14 | Rejected | no score |
| 70Qwen 3.7 PlusAlibaba | 16.35%± 2.57 | $0.30 | Not yet assessed | no score |
| 71=Kimi K2.5Moonshot AI | 15.87%± 2.54 | $0.23 | Passes | 37 |
| 71=Kimi K2.6Moonshot AI | 15.87%± 2.54 | $1.52 | Rejected | no score |
| 71=MiMo V2.5 ProXiaomi | 15.87%± 2.54 | $0.08 | Rejected | no score |
| 74=Grok 4.3xAI (SpaceXAI) | 15.38%± 2.51 | $0.20 | Rejected | no score |
| 74=Nemotron 3 UltraNVIDIA | 15.38%± 2.51 | - | Rejected | no score |
| 76Qwen 3.6 PlusAlibaba | 14.90%± 2.48 | $0.70 | Not yet assessed | no score |
| 77=Gemini 3.5 Flash LiteGoogle | 13.94%± 2.41 | $0.12 | Passes | 38 |
| 77=Grok 4.20 (Reasoning)xAI (SpaceXAI) | 13.94%± 2.41 | $0.26 | Rejected | no score |
| 79GPT 5.4 MiniOpenAI | 12.50%± 2.30 | $1.86 | Passes | 25 |
| 80=Claude Haiku 4.5 (Thinking)Anthropic | 10.58%± 2.14 | $0.27 | Passes | 32 |
| 80=MiniMax-M2.7MiniMax | 10.58%± 2.14 | $0.22 | Rejected | no score |
| 82Nemotron 3.5 Super VLNVIDIA | 10.10%± 2.09 | $0.06 | Rejected | no score |
| 83=MiMo V2.5Xiaomi | 9.13%± 2.00 | $0.03 | Rejected | no score |
| 83=Mistral Medium 3.5Mistral AI | 9.13%± 2.00 | $1.31 | Passes | 23 |
| 85GPT 5.4 NanoOpenAI | 6.25%± 1.68 | $0.14 | Passes | 29 |
| 86Mercury 2.5Inception Labs | 4.33%± 1.41 | $0.06 | Not yet assessed | no score |
| 87Gemini 3.1 Flash Lite PreviewGoogle | 3.37%± 1.25 | $0.01 | Not yet assessed | no score |
| 88Nemotron 3.5 LightningNVIDIA | 2.88%± 1.16 | < $0.01 | Rejected | no score |
| 89Laguna M.1Poolside | 2.40%± 1.06 | $0.22 | Rejected | no score |
| 90Laguna XS.2Poolside | 0.96%± 0.68 | $0.11 | Rejected | no score |
| 91Ling 3.0 FlashAnt Group | 0.00%± 0.00 | $0.04 | Rejected | no score |
Source: Vals AI (Legal Research Bench) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.
Vals AI · Version 1 · Measured 9 November 2026 · Go to the source. The PrudentScore is a composition by Prudai B.V. based on publicly published figures and does not represent the view or endorsement of Vals AI, Harvey or Artificial Analysis. Figures from Artificial Analysis appear in running text only, with attribution and a link, as that party's terms require [9].
Other benchmarks
- Harvey LAB (Vals implementation)
- LegalBench (Vals leaderboard)
- CaseLaw v2
- TaxEval v2
- Tax Agent Bench
- CorpFin v2
- Immigration Bench
- Access to Justice
- Public Benefits Bench
- Vals Legal Index
- Vals Index
- GPQA Diamond (Vals-run)
- MMLU Pro (Vals-run)
- Artificial Analysis Intelligence Index
Claude and Anthropic, GPT and OpenAI, Gemini and Google, Grok and xAI, Kimi and Moonshot AI, Muse and Meta, Mistral, DeepSeek, GLM and Z.ai, Harvey, Vals AI and Artificial Analysis are trademarks of their respective owners. They are named for identification only and this implies no partnership, sponsorship or endorsement.