AutoBench Agentic Run 1 - April 2026

The first AutoBench run to measure agentic performance of top LLMs

Latest

Date

April 19, 2026

Version

2026-04-19

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

AutoBench features 84.56% correlation with GDPval-AA, 68.07% with Tau2-Bench Telecom, 85.15% with Artificial Analysis Intelligence Index, and 83% with Terminal-Bench Hard. Models sorted by AutoBench score.

Model	AutoBench	AAI Index	Terminal-bench	GDPval-AA	Tau2-Bench Telecom
Llama 4 Maverick	2.27 (#32)	7 (#31)	7 (#31)	0 (#31)	18 (#31)
Gemma 4 26B A4B IT	2.61 (#31)	32 (#25)	14 (#28)	26 (#21)	44 (#26)
Mistral large 2512	2.62 (#30)	22 (#29)	16 (#27)	18 (#26)	41 (#27)
Deepseek v3.2	2.64 (#29)	53 (#14)	36 (#13)	35 (#11)	91 (#12)
Gpt oss 20b	2.65 (#28)	28 (#26)	11 (#30)	8 (#29)	60 (#24)
Nova 2 lite v1	2.66 (#27)	37 (#24)	17 (#26)	17 (#28)	73 (#20)
Mistral Small 4	2.69 (#26)	26 (#28)	17 (#25)	18 (#27)	25 (#30)
Nemotron 3 Nano 30B A3B	2.71 (#25)	19 (#30)	14 (#29)	4 (#30)	41 (#28)
Gpt oss 120b	2.76 (#24)	38 (#23)	24 (#23)	22 (#23)	66 (#22)
GPT-5.4 Nano (xhigh)	2.78 (#23)	48 (#18)	42 (#8)	34 (#15)	76 (#19)
MiniMax M2.5	2.79 (#22)	56 (#11)	35 (#16)	34 (#16)	95 (#6)
Gemma 4 31B IT	2.79 (#21)	41 (#20)	36 (#14)	31 (#17)	60 (#23)
Nemotron 3 Super 120B A12B	2.8 (#20)	40 (#22)	29 (#19)	25 (#22)	68 (#21)
Gemini 3.1 Flash Lite Preview	2.82 (#19)	26 (#27)	24 (#22)	21 (#24)	31 (#29)
Qwen3.5 35B A3B	2.82 (#18)	44 (#19)	27 (#21)	21 (#25)	94 (#8)
Qwen3.5 122B A10B	2.84 (#17)	53 (#15)	31 (#18)	31 (#18)	89 (#13)
Grok 4.1 fast	2.84 (#16)	49 (#17)	24 (#24)	27 (#19)	93 (#9)
GPT-5.4 Mini (xhigh)	2.91 (#15)	59 (#9)	52 (#4)	46 (#6)	83 (#16)
GLM 4.7	2.92 (#14)	55 (#12)	32 (#17)	35 (#13)	96 (#3)
Gemini 3 Flash Preview	2.98 (#13)	50 (#16)	39 (#10)	35 (#12)	80 (#17)
Claude haiku 4.5	2.99 (#12)	40 (#21)	27 (#20)	34 (#14)	55 (#25)
Grok 4.20	3 (#11)	54 (#13)	38 (#12)	27 (#20)	93 (#10)
MiniMax M2.7	3.01 (#10)	61 (#7)	39 (#11)	51 (#5)	85 (#15)
Kimi K2.5	3.02 (#9)	59 (#10)	35 (#15)	39 (#10)	96 (#4)
Qwen3.6 Plus	3.07 (#8)	62 (#6)	44 (#6)	43 (#8)	95 (#7)
Mimo V2 Pro	3.1 (#7)	63 (#5)	41 (#9)	46 (#7)	95 (#5)
GPT-5.4 (xhigh)	3.13 (#6)	68 (#2)	58 (#1)	59 (#1)	87 (#14)
GLM 5.1	3.15 (#5)	67 (#3)	43 (#7)	52 (#4)	98 (#1)
Claude Sonnet 4.6	3.16 (#4)	63 (#4)	53 (#3)	58 (#2)	76 (#18)
Gemini 3.1 Pro Preview	3.21 (#3)	59 (#8)	54 (#2)	41 (#9)	96 (#2)
Claude Opus 4.6	3.24 (#2)	68 (#1)	46 (#5)	56 (#3)	92 (#11)
Claude Opus 4.7	3.3 (#1)	-	-	-	-