LLM Benchmarks

Graduate-level multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult.

1

Gemini 3.1 Pro Preview

94.1

2

GPT-5.4

92.0

3

GPT-5.3 Codex

91.5

4

Gemini 3 Pro Preview

90.8

Humanity's Last Exam — extremely challenging questions designed to test the upper limits of AI capability across diverse domains.

1

Gemini 3.1 Pro Preview

44.7

2

GPT-5.4

41.6

3

GPT-5.3 Codex

39.9

4

Gemini 3 Pro Preview

37.2

Science coding benchmark measuring AI ability to solve scientific computing tasks.

1

Gemini 3.1 Pro Preview

58.9

2

GPT-5.4

56.6

3

Gemini 3 Pro Preview

56.1

4

GPT-5.2-Codex

54.6

Instruction Following Benchmark measuring LLM ability to adhere to nuanced writing constraints and formatting requirements.

1

A

Qwen3.5 397B A17B

78.8

2

Gemini 3 Flash Preview

78.0

3

GPT-5.2-Codex

77.6

4

Gemini 3.1 Flash Lite Preview

77.2

Long Context Retrieval benchmark testing ability to find and use information in very long documents.

1

GPT-5.2-Codex

75.7

2

GPT-5

75.6

3

GPT-5.1

75.0

4

K

KAT-Coder-Pro V1

74.0

Tau2 benchmark testing multi-turn agent capabilities in airline and retail domains.

Terminal-based benchmark testing AI ability to interact with command-line interfaces and solve system tasks.

1

GPT-5.4

57.6

2

Gemini 3.1 Pro Preview

53.8

3

Claude Sonnet 4.6

53.0

4

GPT-5.3 Codex

53.0

Real-world coding benchmark with problems from competitive programming contests, testing code generation and problem-solving abilities.

1

Gemini 3 Pro Preview

91.7

2

Gemini 3 Flash Preview

90.8

3

DeepSeek V3.2 Speciale

89.6

4

GPT-5.2

89.4

Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains.

1

Gemini 3 Pro Preview

89.8

2

Gemini 3 Pro Preview

89.5

3

Claude Opus 4.5

89.5

4

Gemini 3 Flash Preview

89.0

American Invitational Mathematics Examination 2025 problems testing olympiad-level mathematical reasoning.

1

GPT-5.2 Pro

99.0

2

GPT-5 Codex

98.7

3

Gemini 3 Flash Preview

97.0

4

DeepSeek V3.2 Speciale

96.7

AGIEval English — human-level reasoning tasks from standardized exams like SAT, LSAT, and civil service exams.

1

Gemini 3.1 Pro Preview

94.0

2

Gemini 3 Pro Preview

93.2

3

Gemini 3 Pro Preview

93.2

4

A

Qwen3.5 397B A17B

91.4

Competition mathematics problems requiring multi-step reasoning, covering algebra, geometry, number theory, and calculus.

1

GPT-5

99.4

2

o3

99.2

3

Grok 3 Mini

99.2

4

Claude Sonnet 4

99.1

American Invitational Mathematics Examination 2024 problems testing olympiad-level mathematical reasoning.

1

GPT-5

95.7

2

Grok 4

94.3

3

o4 Mini

94.0

4

A

Qwen3 235B A22B Thinking 2507

94.0

Big-Bench Hard — challenging subset of BIG-Bench focusing on tasks where language models previously underperformed.

1

Claude Sonnet 4.5

94.4

2

Claude Sonnet 4.5

94.3

3

Z

GLM 5

94.3

4

Z

GLM 5

94.3

OpenAI HumanEval benchmark measuring Python code generation from function docstrings.

1

Claude Sonnet 4.5

97.6

2

R1

97.4

3

Grok 4

97.0

4

Claude Sonnet 4.5

97.0

Accounting and audit benchmark testing financial reasoning capabilities.

1

Claude 3.7 Sonnet

86.7

2

Gemini 2.5 Pro Preview 05-06

86.7

3

Claude 3.7 Sonnet

83.3

4

A

QwQ 32B

83.3

Mostly Basic Python Problems Plus — tests Python code generation with enhanced test cases.

1

A

Qwen3 235B A22B

66.2

2

A

Qwen3 235B A22B

66.2

3

R1

64.7

4

o4 Mini High

64.3

Multimodal Understanding benchmark testing vision-language models on expert-level tasks.

1

o4 Mini High

79.2

2

GPT-5

79.1

3

GPT-5

79.1

4

GPT-5

79.1

Software Engineering benchmark testing ability to resolve real GitHub issues.

1

Claude Opus 4.6

62.7

2

Claude Opus 4.6

62.7

3

MiniMax M2.5

56.3

4

GPT-5

54.3

Grade School Math 8K — 8,500 high quality grade school math word problems.

1

Claude Opus 4

96.2

2

Claude Opus 4

96.2

3

R1

96.2

4

o4 Mini High

96.0

Simple question answering benchmark testing factual accuracy and knowledge retrieval.

1

Gemini 2.5 Pro

53.0

2

A

Qwen3 235B A22B Instruct 2507

50.6

3

A

Qwen3 VL 235B A22B Instruct

46.7

4

GPT-4.1

40.4

AI2 Reasoning Challenge (Challenge set) — grade-school science questions requiring complex reasoning.

1

GPT-5

96.3

2

GPT-5

96.3

3

GPT-5

96.3

4

GPT-5

96.3

AI2 Reasoning Challenge (Easy set) — grade-school science questions.

1

Claude Opus 4

99.7

2

Claude Opus 4

99.7

3

A

Qwen3 32B

99.1

4

A

Qwen3 32B

99.1

Big Bench Extra Hard — even more challenging reasoning tasks pushing the limits of language model capabilities.

1

GPT-5

64.1

2

GPT-5

64.1

3

GPT-5

64.1

4

GPT-5

64.1

Massive Multitask Language Understanding — tests knowledge across 57 subjects.

1

Z

GLM 5

91.7

2

Z

GLM 5

91.7

3

R1 0528

90.5

4

Grok 3 Mini Beta

89.2

Mathematics benchmark covering algebra, geometry, number theory, and calculus problems.

1

Claude Opus 4.6

95.6

2

Claude Opus 4.6

95.6

3

o4 Mini High

94.6

4

Z

GLM 5

94.0

Medical question answering benchmark from USMLE-style questions.

1

o4 Mini High

95.2

2

Gemini 2.5 Pro

94.6

3

Claude 3.7 Sonnet

92.3

4

R1

92.1

AGIEval Chinese — reasoning tasks from Chinese standardized exams (Gaokao, civil service).

1

DeepSeek V3.2 Exp

90.1

2

A

Qwen3 235B A22B

89.4

3

A

Qwen3 235B A22B

89.4

4

ERNIE 4.5 300B A47B

89.0

BIRD-CRITIC — multi-turn benchmark testing SQL generation and database interaction.

1

Claude Opus 4.6

34.0

2

Claude Opus 4.6

34.0

3

Z

GLM 4.7

33.0

4

Z

GLM 4.7

33.0

Stock market benchmark testing financial analysis capabilities.

1

A

Qwen2.5 72B Instruct

100.0

2

R1

91.7

3

GPT-4.1

91.7

4

o3 Mini

83.3

Instruction Following Evaluation benchmark testing how well LLMs follow detailed formatting and content constraints.

1

K

Kimi K2.5

92.6

2

K

Kimi K2.5

92.6

3

Gemini 2.5 Pro

90.8

4

Z

GLM 4.7

90.8

Logic puzzle benchmark based on knights (truth-tellers) and knaves (liars) puzzles.

1

o3 Mini

99.7

2

o4 Mini High

99.7

3

R1 0528

97.9

4

R1

97.3

Berkeley Function Calling Leaderboard v3 — testing function/tool calling accuracy.

Extended formal logic benchmark testing deductive and propositional reasoning.

1

o3 Mini

99.8

2

R1 0528

99.2

3

R1

98.4

4

Claude 3.7 Sonnet

95.6

Weapons of Mass Destruction Proxy — benchmark testing knowledge safety boundaries.

1

Gemini 3 Flash Preview

86.8

2

Gemini 3 Flash Preview

86.8

3

o3 Mini

80.5

4

Claude 3.7 Sonnet

78.2

DarkBench — benchmark testing model safety and resistance to adversarial attacks.

1

Grok 4

51.1

2

Phi 4

49.5

3

A

Qwen3 32B

49.2

4

A

Qwen3 32B

49.2

GAIA — General AI Assistants benchmark testing multi-step real-world tasks.

1

GPT-5 Mini

44.8

2

GPT-5 Mini

44.8

3

Claude 3.7 Sonnet

43.9

4

Claude 3.7 Sonnet

43.9

Workshop on Machine Translation 2014 — multilingual translation quality benchmark.

1

Gemini 2.0 Flash

38.9

2

Llama 3.1 405B Instruct

38.0

3

Llama 4 Maverick

38.0

4

GPT-4.1

37.6

MultiChallenge — general knowledge benchmark with diverse challenge types.

1

Mistral Medium 3.1

37.4

2

Nova Pro 1.0

19.0