AutoBench Agentic Run 1 - April 2026

The first AutoBench run to measure agentic performance of top LLMs

Past

Date

April 16, 2026

Version

2026-04-16

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

AutoBench features 81.81% correlation with GDPval-AA, 66.45% with Tau2-Bench Telecom, 82.71% with Artificial Analysis Intelligence Index, and 80.25% with Terminal-Bench Hard. Models sorted by AutoBench score.

Model	AutoBench	AAI Index	Terminal-bench	GDPval-AA	Tau2-Bench Telecom
Llama 4 Maverick	2.27 (#31)	7 (#31)	7 (#31)	0 (#31)	18 (#31)
Gemma 4 26B A4B IT	2.53 (#30)	32 (#25)	14 (#28)	26 (#21)	44 (#26)
Deepseek v3.2	2.55 (#29)	53 (#14)	36 (#13)	35 (#11)	91 (#12)
Mistral large 2512	2.62 (#28)	22 (#29)	16 (#27)	18 (#26)	41 (#27)
Nemotron 3 Nano 30B A3B	2.63 (#27)	19 (#30)	14 (#29)	4 (#30)	41 (#28)
Gpt oss 20b	2.65 (#26)	28 (#26)	11 (#30)	8 (#29)	60 (#24)
Nova 2 lite v1	2.66 (#25)	37 (#24)	17 (#26)	17 (#28)	73 (#20)
Mistral Small 4	2.69 (#24)	26 (#28)	17 (#25)	18 (#27)	25 (#30)
Gemma 4 31B IT	2.7 (#22)	41 (#20)	36 (#14)	31 (#17)	60 (#23)
Nemotron 3 Super 120B A12B	2.7 (#23)	40 (#22)	29 (#19)	25 (#22)	68 (#21)
MiniMax M2.5	2.72 (#21)	56 (#11)	35 (#16)	34 (#16)	95 (#6)
Gpt oss 120b	2.76 (#20)	38 (#23)	24 (#23)	22 (#23)	66 (#22)
GPT-5.4 Nano (xhigh)	2.78 (#19)	48 (#18)	42 (#8)	34 (#15)	76 (#19)
Gemini 3.1 Flash Lite Preview	2.82 (#18)	26 (#27)	24 (#22)	21 (#24)	31 (#29)
Qwen3.5 35B A3B	2.83 (#17)	44 (#19)	27 (#21)	21 (#25)	94 (#8)
Qwen3.5 122B A10B	2.84 (#15)	53 (#15)	31 (#18)	31 (#18)	89 (#13)
Grok 4.1 fast	2.84 (#16)	49 (#17)	24 (#24)	27 (#19)	93 (#9)
Gemini 3 Flash Preview	2.89 (#14)	50 (#16)	39 (#10)	35 (#12)	80 (#17)
MiniMax M2.7	2.9 (#13)	61 (#7)	39 (#11)	51 (#5)	85 (#15)
GPT-5.4 Mini (xhigh)	2.91 (#12)	59 (#9)	52 (#4)	46 (#6)	83 (#16)
GLM 4.7	2.92 (#9)	55 (#12)	32 (#17)	35 (#13)	96 (#3)
Grok 4.20	2.92 (#10)	54 (#13)	38 (#12)	27 (#20)	93 (#10)
Kimi K2.5	2.92 (#8)	59 (#10)	35 (#15)	39 (#10)	96 (#4)
Claude haiku 4.5	2.92 (#11)	40 (#21)	27 (#20)	34 (#14)	55 (#25)
Mimo V2 Pro	2.99 (#6)	63 (#5)	41 (#9)	46 (#7)	95 (#5)
Qwen3.6 Plus	2.99 (#7)	62 (#6)	44 (#6)	43 (#8)	95 (#7)
GPT-5.4 (xhigh)	3.02 (#5)	68 (#2)	58 (#1)	59 (#1)	87 (#14)
GLM 5.1	3.06 (#4)	67 (#3)	43 (#7)	52 (#4)	98 (#1)
Gemini 3.1 Pro Preview	3.1 (#3)	59 (#8)	54 (#2)	41 (#9)	96 (#2)
Claude Sonnet 4.6	3.13 (#2)	63 (#4)	53 (#3)	58 (#2)	76 (#18)
Claude Opus 4.6	3.17 (#1)	68 (#1)	46 (#5)	56 (#3)	92 (#11)