AutoBench Agentic Run 1 - April 2026

The first AutoBench run to measure agentic performance of top LLMs

Past

Date

April 16, 2026

Version

2026-04-16

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

AutoBench provides a complete dashboard with highly customizable domain-specific average ranks and precise efficiency metrics (average answer cost, average answer duration, P99 Latency). For further detail also check our Hugging Face Leaderboard.

Model	Score	Avg Cost ($ Cents)	Avg Latency (sec)	P99 Latency (sec)	Iterations
Claude Opus 4.6	3.17 (#1)	2.56 (#30)	38s (#14)	97s (#13)	198
Deepseek v3.2	2.55 (#29)	0.06 (#8)	54s (#22)	156s (#21)	198
Gemini 3 Flash Preview	2.89 (#14)	0.28 (#20)	13s (#3)	26s (#2)	198
Gemini 3.1 Pro Preview	3.10 (#3)	1.33 (#25)	26s (#10)	198s (#25)	198
Mimo V2 Pro	2.99 (#7)	0.32 (#21)	26s (#9)	54s (#7)	198
Qwen3.5 122B A10B	2.84 (#16)	0.14 (#18)	13s (#4)	36s (#4)	198
Qwen3.5 35B A3B	2.83 (#17)	0.11 (#13)	14s (#5)	31s (#3)	198
Gpt oss 120b	2.76 (#20)	0.02 (#2)	18s (#7)	63s (#9)	198
Claude Sonnet 4.6	3.13 (#2)	1.95 (#28)	46s (#19)	147s (#19)	197
GLM 5.1	3.06 (#4)	0.50 (#23)	66s (#25)	177s (#24)	197
Gpt oss 20b	2.65 (#26)	0.01 (#1)	43s (#16)	166s (#22)	197
Qwen3.6 Plus	2.99 (#6)	0.20 (#19)	48s (#20)	109s (#15)	197
Grok 4.1 fast	2.84 (#15)	0.12 (#15)	36s (#12)	96s (#12)	197
Claude haiku 4.5	2.92 (#10)	0.86 (#24)	44s (#18)	152s (#20)	195
Llama 4 Maverick	2.27 (#31)	0.03 (#5)	41s (#15)	76s (#11)	195
GLM 4.7	2.92 (#9)	0.14 (#17)	44s (#17)	44s (#6)	194
Mistral Small 4	2.69 (#24)	0.05 (#6)	11s (#2)	41s (#5)	194
MiniMax M2.7	2.90 (#13)	0.09 (#11)	28s (#11)	68s (#10)	193
Mistral large 2512	2.62 (#28)	0.10 (#12)	9s (#1)	22s (#1)	193
MiniMax M2.5	2.72 (#21)	0.05 (#7)	85s (#27)	241s (#27)	192
Nemotron 3 Nano 30B A3B	2.63 (#27)	0.08 (#10)	103s (#30)	279s (#30)	189
Grok 4.20	2.92 (#11)	1.54 (#26)	37s (#13)	98s (#14)	187
Gemma 4 26B A4B IT	2.53 (#30)	0.02 (#3)	14s (#6)	114s (#16)	186
Nemotron 3 Super 120B A12B	2.70 (#23)	0.07 (#9)	72s (#26)	245s (#28)	183
Kimi K2.5	2.92 (#8)	0.13 (#16)	55s (#23)	174s (#23)	182
Nova 2 lite v1	2.66 (#25)	1.56 (#27)	57s (#24)	139s (#18)	181
Gemini 3.1 Flash Lite Preview	2.82 (#18)	0.11 (#14)	23s (#8)	135s (#17)	179
Gemma 4 31B IT	2.70 (#22)	0.02 (#4)	52s (#21)	58s (#8)	169
GPT-5.4 (xhigh)	3.02 (#5)	5.82 (#31)	129s (#31)	329s (#31)	132
GPT-5.4 Nano (xhigh)	2.78 (#19)	0.43 (#22)	93s (#29)	262s (#29)	113
GPT-5.4 Mini (xhigh)	2.91 (#12)	2.03 (#29)	87s (#28)	241s (#26)	104