Benchmark results
See the tests behind each score. They use the benchmark’s original scoring scale.
Chat1 result
IFEvalifeval · chat
90.4%
ReportedOct 2026
Factuality1 result
SimpleQAsimpleqa · factuality
10.0%
ReportedOct 2026
General intelligence1 result
Global-MMLU-Liteglobal-mmlu-lite · general
75.1%
ReportedOct 2026
Language3 results
ECLeKTiceclektic · language
16.7%
ReportedOct 2026
MMLU-Prommlu-pro · language
67.5%
ReportedOct 2026
WMT24++wmt24++ · language
53.4%
ReportedOct 2026
Long Context1 result
MRCR v2 (8-needle)mrcr-v2-(8-needle) · long_context
13.5%
ReportedOct 2026
Math4 results
GSM8kgsm8k · math
95.9%
ReportedOct 2026
HiddenMathhiddenmath · math
60.3%
ReportedOct 2026
MATHmath · math
89.0%
ReportedOct 2026
MathVista-Minimathvista-mini · math
67.6%
ReportedOct 2026
Reasoning10 results
BIG-Bench Extra Hardbig-bench-extra-hard · reasoning
19.3%
ReportedOct 2026
BIG-Bench Hardbig-bench-hard · reasoning
87.6%
ReportedOct 2026
Bird-SQL (dev)bird-sql-(dev) · reasoning
54.4%
ReportedOct 2026
ChartQAchartqa · reasoning
78.0%
ReportedOct 2026
FACTS Groundingfacts-grounding · reasoning
74.9%
ReportedOct 2026
GPQAgpqa · reasoning
42.4%
ReportedOct 2026
HumanEvalhumaneval · reasoning
87.8%
ReportedOct 2026
LiveCodeBenchlivecodebench · reasoning
29.7%
ReportedOct 2026
MBPPmbpp · reasoning
74.4%
ReportedOct 2026
Natural2Codenatural2code · reasoning
84.5%
ReportedOct 2026
Vision6 results
AI2Dai2d · vision
84.5%
ReportedOct 2026
DocVQAdocvqa · vision
86.6%
ReportedOct 2026
InfoVQAinfovqa · vision
70.6%
ReportedOct 2026
MMMU (val)mmmu-(val) · vision
64.9%
ReportedOct 2026
TextVQAtextvqa · vision
65.1%
ReportedOct 2026
VQAv2 (val)vqav2-(val) · vision
71.0%
ReportedOct 2026
Learn what each benchmark measures ↗