AutoBench Agentic Run 1 - April 2026

The first AutoBench run to measure agentic performance of top LLMs

Latest

Date

April 19, 2026

Version

2026-04-19

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

AutoBench provides a complete dashboard with highly customizable domain-specific average ranks and precise efficiency metrics (average answer cost, average answer duration, P99 Latency). For further detail also check our Hugging Face Leaderboard.

Model	Score	Avg Cost ($ Cents)	Avg Latency (sec)	P99 Latency (sec)	Iterations
GPT-5.4 (xhigh)	3.13 (#6)	6.33 (#32)	131s (#32)	326s (#32)	145
Claude Opus 4.7	3.30 (#1)	2.72 (#31)	21s (#8)	47s (#7)	187
Claude Opus 4.6	3.24 (#2)	2.58 (#30)	38s (#15)	98s (#15)	198
GPT-5.4 Mini (xhigh)	2.91 (#15)	2.03 (#29)	87s (#29)	241s (#28)	104
Claude Sonnet 4.6	3.16 (#4)	1.98 (#28)	47s (#22)	149s (#21)	193
Nova 2 lite v1	2.66 (#27)	1.56 (#27)	57s (#25)	139s (#20)	181
Grok 4.20	3.00 (#11)	1.52 (#26)	33s (#13)	79s (#13)	189
Gemini 3.1 Pro Preview	3.21 (#3)	1.33 (#25)	26s (#11)	58s (#10)	198
Claude haiku 4.5	2.99 (#12)	0.84 (#24)	43s (#18)	151s (#22)	193
GLM 5.1	3.15 (#5)	0.51 (#23)	60s (#26)	183s (#26)	197
GPT-5.4 Nano (xhigh)	2.78 (#23)	0.43 (#22)	93s (#30)	262s (#30)	113
Mimo V2 Pro	3.10 (#7)	0.33 (#21)	26s (#10)	55s (#8)	199
Gemini 3 Flash Preview	2.98 (#13)	0.28 (#20)	13s (#4)	23s (#2)	198
Qwen3.6 Plus	3.07 (#8)	0.19 (#19)	46s (#21)	107s (#16)	198
Qwen3.5 122B A10B	2.84 (#17)	0.14 (#18)	13s (#5)	36s (#4)	198
GLM 4.7	2.92 (#14)	0.14 (#17)	44s (#19)	135s (#19)	194
Kimi K2.5	3.02 (#9)	0.13 (#16)	54s (#24)	178s (#25)	187
Grok 4.1 fast	2.84 (#16)	0.12 (#15)	36s (#14)	96s (#14)	197
Gemini 3.1 Flash Lite Preview	2.82 (#19)	0.12 (#14)	23s (#9)	114s (#17)	179
Qwen3.5 35B A3B	2.82 (#18)	0.11 (#13)	14s (#6)	31s (#3)	198
Mistral large 2512	2.62 (#30)	0.10 (#12)	9s (#1)	22s (#1)	193
MiniMax M2.7	3.01 (#10)	0.10 (#11)	27s (#12)	58s (#9)	193
Nemotron 3 Nano 30B A3B	2.71 (#25)	0.08 (#10)	106s (#31)	285s (#31)	190
Nemotron 3 Super 120B A12B	2.80 (#20)	0.06 (#9)	69s (#27)	245s (#29)	187
Deepseek v3.2	2.64 (#29)	0.06 (#8)	54s (#23)	129s (#18)	197
MiniMax M2.5	2.79 (#22)	0.05 (#7)	75s (#28)	241s (#27)	184
Mistral Small 4	2.69 (#26)	0.05 (#6)	11s (#2)	41s (#6)	194
Llama 4 Maverick	2.27 (#32)	0.03 (#5)	41s (#16)	76s (#12)	195
Gemma 4 31B IT	2.79 (#21)	0.02 (#4)	45s (#20)	174s (#24)	183
Gemma 4 26B A4B IT	2.61 (#31)	0.02 (#3)	12s (#3)	41s (#5)	192
Gpt oss 120b	2.76 (#24)	0.02 (#2)	18s (#7)	63s (#11)	198
Gpt oss 20b	2.65 (#28)	0.01 (#1)	43s (#17)	166s (#23)	197