Skip to content

Benchmarks · Vals AI

Legal Research Bench

Note: the questions cover US law; Vals has no benchmark for Dutch or EU law. Measures whether an AI agent with search tools fully answers a legal research question correctly: 208 test questions (private test set, from 413 expert-authored, peer-reviewed questions) across eight practice areas, graded by GPT-5.4 as judge against a rubric; "all-pass" counts only when every criterion is met.

Models measured
91
Top
55.77 %Gemini 4 Carbonwithin the standard error of ±3.45
Version
1
Measured
9 November 2026
Go to the sourceIn the PrudentScore, version 1 of this benchmark counts for 50 percent.

01

Ranking

At the top: Top 15

Vals AI Legal Research Bench, all-pass [1]. Hatched bars are rejected by the compliance gate.

  1. 1Gemini 4 Carbon55.77%
  2. 2=Claude Fable 5.155.29%
  3. 2=Claude Opus 555.29%
  4. 2=Muse Spark 1.3 (Max)55.29%
  5. 5Gemini 4 Argon54.81%
  6. 6Wonder Full53.37%
  7. 7=Claude Opus 5.550.48%
  8. 7=Wonder50.48%
  9. 9Claude Fable 549.52%
  10. 10=GLM 5.349.04%
  11. 10=Muse Ember49.04%
  12. 12=Claude Sonnet 5.548.08%
  13. 12=GPT-5.6 Sol48.08%
  14. 12=Grok 4.648.08%
  15. 15Qwen 3.8 Max47.60%

Source: Vals AI (Legal Research Bench) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.

Passes the compliance gateRejected by the compliance gate
Quality against cost per taskEach dot is a model: cost per completed task in dollars (logarithmic) against all-pass on legal research [1]. Top left is cheap and good. The dashed line is the Pareto frontier of the models that pass the gate: no other model that passes the gate is both cheaper and better. Click a dot for the model page.
Passes the compliance gateRejected by the compliance gatePareto frontier (gate passed)

Hover a dot for the figures. Models without a published cost per task are not in this chart.

Cost per completed task against the legal-research score: Gemini 4 Argon: $6.45, 54.81%, Passes the compliance gate, on the Pareto frontier. Gemini 4 Carbon: $8.74, 55.77%, Passes the compliance gate, on the Pareto frontier. Claude Opus 5: $6.76, 55.29%, Passes the compliance gate, on the Pareto frontier. Claude Opus 5.5: $23.61, 50.48%, Passes the compliance gate. Claude Opus 4.8: $2.82, 43.75%, Passes the compliance gate. Gemini 3.8 Flash: $1.63, 38.94%, Passes the compliance gate, on the Pareto frontier. Claude Sonnet 5.5: $13.66, 48.08%, Passes the compliance gate. Claude Haiku 5.5: $1.65, 43.75%, Passes the compliance gate, on the Pareto frontier. Claude Sonnet 5: $2.72, 41.83%, Passes the compliance gate. GPT-6.1 Sol: $3.38, 38.46%, Passes the compliance gate. Gemini 3.7 Flash: $0.76, 34.62%, Passes the compliance gate, on the Pareto frontier. GPT-5.6 Sol: $21.61, 48.08%, Passes the compliance gate. GPT-5.6 Luna: $0.84, 36.54%, Passes the compliance gate, on the Pareto frontier. GPT-6 Astra: $10.58, 39.42%, Passes the compliance gate. Claude Sonnet 4.6: $2.30, 38.46%, Passes the compliance gate. Claude Opus 4.7: $3.06, 38.46%, Passes the compliance gate. GPT-6 Luna: $0.44, 30.29%, Passes the compliance gate, on the Pareto frontier. GPT-5.6 Terra: $7.91, 41.35%, Passes the compliance gate. GPT 5.5: $7.40, 40.38%, Passes the compliance gate. Gemini 3.5 Flash: $1.31, 30.77%, Passes the compliance gate. Gemini 3.6 Flash: $0.47, 25.00%, Passes the compliance gate. GPT-6 Sol: $4.90, 28.85%, Passes the compliance gate. GPT-6.1 Terra: $3.22, 25.96%, Passes the compliance gate. Gemini 3.5 Flash Lite: $0.12, 13.94%, Passes the compliance gate, on the Pareto frontier. Kimi K2.5: $0.23, 15.87%, Passes the compliance gate, on the Pareto frontier. Claude Haiku 4.5 (Thinking): $0.27, 10.58%, Passes the compliance gate. GPT 5.4 Nano: $0.14, 6.25%, Passes the compliance gate. GPT 5.4 Mini: $1.86, 12.50%, Passes the compliance gate. Mistral Medium 3.5: $1.31, 9.13%, Passes the compliance gate. Claude Fable 5.1: $23.06, 55.29%, Rejected by the compliance gate. Muse Spark 1.3 (Max): $0.59, 55.29%, Rejected by the compliance gate. Wonder Full: $3.60, 53.37%, Rejected by the compliance gate. Wonder: $3.98, 50.48%, Rejected by the compliance gate. Claude Fable 5: $9.79, 49.52%, Rejected by the compliance gate. GLM 5.3: $2.24, 49.04%, Rejected by the compliance gate. Muse Ember: $0.06, 49.04%, Rejected by the compliance gate. Grok 4.6: $1.53, 48.08%, Rejected by the compliance gate. Qwen 3.8 Max: $2.49, 47.60%, Gate not yet assessed. Grok 4.7: $4.92, 47.12%, Rejected by the compliance gate. MiMo V2.6 Pro: $0.18, 47.12%, Rejected by the compliance gate. Inkling 2 Small: $0.29, 46.63%, Rejected by the compliance gate. Claude Fable 5.2: $3.96, 46.15%, Rejected by the compliance gate. Kimi K3: $3.47, 46.15%, Rejected by the compliance gate. DeepSeek V4.1 Flash 1017: $0.70, 45.19%, Rejected by the compliance gate. GLM 5.3 Flash: $0.24, 45.19%, Rejected by the compliance gate. Hy4 Preview: $0.85, 45.19%, Rejected by the compliance gate. Ember-1: $3.67, 44.71%, Gate not yet assessed. Muse Spark 1.2: $0.51, 43.75%, Rejected by the compliance gate. GLM Legal: $0.05, 41.83%, Rejected by the compliance gate. DeepSeek V4.1 Flash: $0.25, 41.35%, Rejected by the compliance gate. Hy4: $2.22, 41.35%, Rejected by the compliance gate. DeepSeek V4 Pro (0813): $1.09, 40.87%, Rejected by the compliance gate. Muse Spark 1.3: $0.65, 40.87%, Rejected by the compliance gate. Step 5 Preview: $0.84, 40.38%, Rejected by the compliance gate. Muse Spark Open: $1.56, 38.46%, Rejected by the compliance gate. Grok 4.5: $1.09, 37.98%, Rejected by the compliance gate. MiMo V2.6 Flash: $0.07, 37.98%, Rejected by the compliance gate. Muse Spark 1.1: $0.38, 37.98%, Rejected by the compliance gate. Kimi K3.1 Pro: $7.23, 37.50%, Rejected by the compliance gate. Qwen 3.8 27B: $2.16, 36.06%, Rejected by the compliance gate. Mistral Large 4 (GA, 26.10): $1.09, 35.58%, Gate not yet assessed. MiniMax-M3.1-Flash: $0.44, 34.13%, Rejected by the compliance gate. Mistral Large 4 (preview): $2.72, 31.73%, Rejected by the compliance gate. GLM 5.2: $0.89, 31.25%, Rejected by the compliance gate. DeepSeek V4 Flash 0731: $0.28, 30.29%, Rejected by the compliance gate. MiniMax-M3: $0.34, 29.81%, Rejected by the compliance gate. Mercury 3: $0.07, 28.85%, Gate not yet assessed. Inkling: $0.49, 28.36%, Rejected by the compliance gate. GLM 5.1: $0.39, 27.89%, Rejected by the compliance gate. Inkling Small: $0.14, 25.48%, Rejected by the compliance gate. Qwen 3.7 Max: $1.40, 25.48%, Gate not yet assessed. DeepSeek V4 Pro (2026-04-23): $0.68, 23.08%, Rejected by the compliance gate. Gemini 3.1 Pro Preview (02/26): $1.10, 20.67%, Rejected by the compliance gate. Gemini 3 Flash (12/25): $0.14, 18.27%, Rejected by the compliance gate. Qwen 3.7 Plus: $0.30, 16.35%, Gate not yet assessed. Kimi K2.6: $1.52, 15.87%, Rejected by the compliance gate. MiMo V2.5 Pro: $0.08, 15.87%, Rejected by the compliance gate. Grok 4.3: $0.20, 15.38%, Rejected by the compliance gate. Qwen 3.6 Plus: $0.70, 14.90%, Gate not yet assessed. Grok 4.20 (Reasoning): $0.26, 13.94%, Rejected by the compliance gate. MiniMax-M2.7: $0.22, 10.58%, Rejected by the compliance gate. Nemotron 3.5 Super VL: $0.06, 10.10%, Rejected by the compliance gate. MiMo V2.5: $0.03, 9.13%, Rejected by the compliance gate. Mercury 2.5: $0.06, 4.33%, Gate not yet assessed. Gemini 3.1 Flash Lite Preview: $0.01, 3.37%, Gate not yet assessed. Nemotron 3.5 Lightning: < $0.01, 2.88%, Rejected by the compliance gate. Laguna M.1: $0.22, 2.40%, Rejected by the compliance gate. Laguna XS.2: $0.11, 0.96%, Rejected by the compliance gate. Ling 3.0 Flash: $0.04, 0.00%, Rejected by the compliance gate.

Legal Research Bench 1
# ModelScoreCost/taskGatePrudentScore
1Gemini 4 CarbonGoogle55.77%± 3.45$8.74Passes77
2=Claude Fable 5.1Anthropic55.29%± 3.46$23.06Rejectedno score
2=Claude Opus 5Anthropic55.29%± 3.46$6.76Passes72
2=Muse Spark 1.3 (Max)Meta55.29%± 3.46$0.59Rejectedno score
5Gemini 4 ArgonGoogle54.81%± 3.46$6.45Passes79
6Wonder FullMeta53.37%± 3.47$3.60Rejectedno score
7=Claude Opus 5.5Anthropic50.48%± 3.48$23.61Passes63
7=WonderMeta50.48%± 3.48$3.98Rejectedno score
9Claude Fable 5Anthropic49.52%± 3.48$9.79Rejectedno score
10=GLM 5.3Z.ai49.04%± 3.48$2.24Rejectedno score
10=Muse EmberMeta49.04%± 3.48$0.06Rejectedno score
12=Claude Sonnet 5.5Anthropic48.08%± 3.47$13.66Passes62
12=GPT-5.6 SolOpenAI48.08%± 3.47$21.61Passes57
12=Grok 4.6xAI (SpaceXAI)48.08%± 3.47$1.53Rejectedno score
15Qwen 3.8 MaxAlibaba47.60%± 3.47$2.49Not yet assessedno score
16=Grok 4.7xAI (SpaceXAI)47.12%± 3.47$4.92Rejectedno score
16=MiMo V2.6 ProXiaomi47.12%± 3.47$0.18Rejectedno score
18Inkling 2 SmallThinking Machines46.63%± 3.47$0.29Rejectedno score
19=Claude Fable 5.2Anthropic46.15%± 3.46$3.96Rejectedno score
19=Kimi K3Moonshot AI46.15%± 3.46$3.47Rejectedno score
21=DeepSeek V4.1 Flash 1017DeepSeek45.19%± 3.46$0.70Rejectedno score
21=GLM 5.3 FlashZ.ai45.19%± 3.46$0.24Rejectedno score
21=Hy4 PreviewTencent45.19%± 3.46$0.85Rejectedno score
24Ember-1Fireworks AI44.71%± 3.46$3.67Not yet assessedno score
25=Claude Haiku 5.5Anthropic43.75%± 3.45$1.65Passes59
25=Claude Opus 4.8Anthropic43.75%± 3.45$2.82Passes62
25=Muse Spark 1.2Meta43.75%± 3.45$0.51Rejectedno score
28=Claude Sonnet 5Anthropic41.83%± 3.43$2.72Passes59
28=GLM LegalZ.ai41.83%± 3.43$0.05Rejectedno score
30=DeepSeek V4.1 FlashDeepSeek41.35%± 3.42$0.25Rejectedno score
30=GPT-5.6 TerraOpenAI41.35%± 3.42$7.91Passes53
30=Hy4Tencent41.35%± 3.42$2.22Rejectedno score
33=DeepSeek V4 Pro (0813)DeepSeek40.87%± 3.42$1.09Rejectedno score
33=Muse Spark 1.3Meta40.87%± 3.42$0.65Rejectedno score
35=GPT 5.5OpenAI40.38%± 3.41$7.40Passes51
35=Step 5 PreviewStepFun40.38%± 3.41$0.84Rejectedno score
37GPT-6 AstraOpenAI39.42%± 3.40$10.58Passes56
38Gemini 3.8 FlashGoogle38.94%± 3.39$1.63Passes62
39=Claude Opus 4.7Anthropic38.46%± 3.38$3.06Passes54
39=Claude Sonnet 4.6Anthropic38.46%± 3.38$2.30Passes55
39=GPT-6.1 SolOpenAI38.46%± 3.38$3.38Passes59
39=Muse Spark OpenMeta38.46%± 3.38$1.56Rejectedno score
43=Grok 4.5xAI (SpaceXAI)37.98%± 3.37$1.09Rejectedno score
43=MiMo V2.6 FlashXiaomi37.98%± 3.37$0.07Rejectedno score
43=Muse Spark 1.1Meta37.98%± 3.37$0.38Rejectedno score
46Kimi K3.1 ProMoonshot AI37.50%± 3.37$7.23Rejectedno score
47GPT-5.6 LunaOpenAI36.54%± 3.35$0.84Passes56
48Qwen 3.8 27BAlibaba36.06%± 3.34$2.16Rejectedno score
49Mistral Large 4 (GA, 26.10)Mistral AI35.58%± 3.33$1.09Not yet assessedno score
50Gemini 3.7 FlashGoogle34.62%± 3.31$0.76Passes58
51MiniMax-M3.1-FlashMiniMax34.13%± 3.30$0.44Rejectedno score
52Mistral Large 4 (preview)Mistral AI31.73%± 3.23$2.72Rejectedno score
53GLM 5.2Z.ai31.25%± 3.22$0.89Rejectedno score
54Gemini 3.5 FlashGoogle30.77%± 3.21$1.31Passes47
55=DeepSeek V4 Flash 0731DeepSeek30.29%± 3.19$0.28Rejectedno score
55=GPT-6 LunaOpenAI30.29%± 3.19$0.44Passes54
57MiniMax-M3MiniMax29.81%± 3.18$0.34Rejectedno score
58=Beam 1.0Reflection AI28.85%± 3.15-Rejectedno score
58=GPT-6 SolOpenAI28.85%± 3.15$4.90Passes45
58=Mercury 3Inception Labs28.85%± 3.15$0.07Not yet assessedno score
61InklingThinking Machines28.36%± 3.13$0.49Rejectedno score
62GLM 5.1Z.ai27.89%± 3.12$0.39Rejectedno score
63GPT-6.1 TerraOpenAI25.96%± 3.05$3.22Passes44
64=Inkling SmallThinking Machines25.48%± 3.03$0.14Rejectedno score
64=Qwen 3.7 MaxAlibaba25.48%± 3.03$1.40Not yet assessedno score
66Gemini 3.6 FlashGoogle25.00%± 3.01$0.47Passes46
67DeepSeek V4 Pro (2026-04-23)DeepSeek23.08%± 2.93$0.68Rejectedno score
68Gemini 3.1 Pro Preview (02/26)Google20.67%± 2.81$1.10Rejectedno score
69Gemini 3 Flash (12/25)Google18.27%± 2.69$0.14Rejectedno score
70Qwen 3.7 PlusAlibaba16.35%± 2.57$0.30Not yet assessedno score
71=Kimi K2.5Moonshot AI15.87%± 2.54$0.23Passes37
71=Kimi K2.6Moonshot AI15.87%± 2.54$1.52Rejectedno score
71=MiMo V2.5 ProXiaomi15.87%± 2.54$0.08Rejectedno score
74=Grok 4.3xAI (SpaceXAI)15.38%± 2.51$0.20Rejectedno score
74=Nemotron 3 UltraNVIDIA15.38%± 2.51-Rejectedno score
76Qwen 3.6 PlusAlibaba14.90%± 2.48$0.70Not yet assessedno score
77=Gemini 3.5 Flash LiteGoogle13.94%± 2.41$0.12Passes38
77=Grok 4.20 (Reasoning)xAI (SpaceXAI)13.94%± 2.41$0.26Rejectedno score
79GPT 5.4 MiniOpenAI12.50%± 2.30$1.86Passes25
80=Claude Haiku 4.5 (Thinking)Anthropic10.58%± 2.14$0.27Passes32
80=MiniMax-M2.7MiniMax10.58%± 2.14$0.22Rejectedno score
82Nemotron 3.5 Super VLNVIDIA10.10%± 2.09$0.06Rejectedno score
83=MiMo V2.5Xiaomi9.13%± 2.00$0.03Rejectedno score
83=Mistral Medium 3.5Mistral AI9.13%± 2.00$1.31Passes23
85GPT 5.4 NanoOpenAI6.25%± 1.68$0.14Passes29
86Mercury 2.5Inception Labs4.33%± 1.41$0.06Not yet assessedno score
87Gemini 3.1 Flash Lite PreviewGoogle3.37%± 1.25$0.01Not yet assessedno score
88Nemotron 3.5 LightningNVIDIA2.88%± 1.16< $0.01Rejectedno score
89Laguna M.1Poolside2.40%± 1.06$0.22Rejectedno score
90Laguna XS.2Poolside0.96%± 0.68$0.11Rejectedno score
91Ling 3.0 FlashAnt Group0.00%± 0.00$0.04Rejectedno score

Source: Vals AI (Legal Research Bench) · Licence: CC BY 4.0 · Adaptation: Prudai selected the models, translated the explanations, added the gate verdict and the PrudentScore and rounded figures to two decimals.

Vals AI · Version 1 · Measured 9 November 2026 · Go to the source. The PrudentScore is a composition by Prudai B.V. based on publicly published figures and does not represent the view or endorsement of Vals AI, Harvey or Artificial Analysis. Figures from Artificial Analysis appear in running text only, with attribution and a link, as that party's terms require [9].

Other benchmarks

Claude and Anthropic, GPT and OpenAI, Gemini and Google, Grok and xAI, Kimi and Moonshot AI, Muse and Meta, Mistral, DeepSeek, GLM and Z.ai, Harvey, Vals AI and Artificial Analysis are trademarks of their respective owners. They are named for identification only and this implies no partnership, sponsorship or endorsement.