AutoBench Agentic Run 1 - April 2026

The first AutoBench run to measure agentic performance of top LLMs

Latest

Date

April 19, 2026

Version

2026-04-19

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

Average latency breakdown across 10 specialized domains, measured in seconds. These metrics help you identify models that consistently deliver quick responses for your specific use case while maintaining quality standards.

Model	Average (All Topics)	Adaptive Replanning	Api Workflow	Domain Workflow	Error Handling	Failure Recovery	Multi Step Orchestration	Parallel Execution	Parameter Complexity	Single Tool Call	Tool Selection
GPT-5.4 (xhigh)	131.01s (#32)	181.33s (#32)	117.59s (#31)	108.80s (#31)	140.56s (#31)	86.83s (#28)	96.52s (#29)	117.26s (#31)	196.75s (#32)	113.17s (#32)	174.45s (#32)
Nemotron 3 Nano 30B A3B	105.95s (#31)	128.85s (#29)	126.32s (#32)	119.78s (#32)	117.21s (#30)	102.23s (#32)	140.19s (#31)	127.50s (#32)	78.06s (#29)	52.46s (#29)	67.39s (#30)
GPT-5.4 Nano (xhigh)	93.34s (#30)	128.94s (#30)	68.28s (#27)	67.87s (#24)	145.88s (#32)	77.75s (#26)	140.74s (#32)	82.56s (#29)	78.78s (#30)	80.94s (#31)	78.70s (#31)
GPT-5.4 Mini (xhigh)	86.91s (#29)	153.36s (#31)	90.63s (#30)	81.13s (#28)	90.32s (#28)	101.93s (#31)	103.88s (#30)	49.85s (#19)	91.57s (#31)	78.56s (#30)	26.54s (#17)
MiniMax M2.5	75.15s (#28)	76.03s (#26)	77.74s (#29)	102.61s (#30)	108.86s (#29)	97.95s (#30)	73.68s (#26)	66.32s (#24)	38.84s (#25)	35.18s (#25)	53.12s (#29)
Nemotron 3 Super 120B A12B	69.40s (#27)	100.80s (#28)	63.74s (#25)	82.07s (#29)	79.34s (#27)	88.14s (#29)	83.45s (#28)	69.54s (#26)	33.77s (#21)	37.39s (#27)	44.53s (#25)
GLM 5.1	60.30s (#26)	66.59s (#23)	67.19s (#26)	72.35s (#26)	67.94s (#24)	71.40s (#25)	61.28s (#24)	87.51s (#30)	31.59s (#19)	25.73s (#22)	36.13s (#23)
Nova 2 lite v1	56.85s (#25)	84.84s (#27)	69.59s (#28)	61.18s (#23)	79.18s (#26)	69.15s (#24)	58.65s (#23)	81.88s (#28)	23.93s (#15)	12.66s (#9)	16.30s (#11)
Kimi K2.5	53.61s (#24)	70.44s (#24)	41.20s (#16)	53.05s (#21)	70.31s (#25)	49.94s (#22)	46.83s (#19)	73.56s (#27)	46.91s (#28)	40.05s (#28)	31.13s (#22)
Deepseek v3.2	53.60s (#23)	73.68s (#25)	59.48s (#24)	43.43s (#17)	49.87s (#18)	81.78s (#27)	74.00s (#27)	62.39s (#23)	25.23s (#16)	26.89s (#23)	29.63s (#19)
Claude Sonnet 4.6	47.09s (#22)	50.63s (#21)	54.78s (#23)	79.74s (#27)	49.83s (#17)	51.20s (#23)	41.36s (#16)	53.78s (#21)	44.22s (#27)	18.72s (#15)	29.96s (#20)
Qwen3.6 Plus	46.43s (#21)	57.86s (#22)	40.92s (#15)	59.86s (#22)	58.99s (#20)	41.77s (#18)	43.74s (#17)	58.85s (#22)	22.29s (#13)	31.20s (#24)	40.97s (#24)
Gemma 4 31B IT	45.28s (#20)	39.20s (#15)	47.50s (#21)	70.79s (#25)	50.06s (#19)	43.48s (#20)	51.45s (#21)	69.19s (#25)	20.98s (#12)	19.60s (#16)	45.34s (#27)
GLM 4.7	43.55s (#19)	50.45s (#20)	40.41s (#14)	43.28s (#16)	66.69s (#23)	39.12s (#17)	48.02s (#20)	52.61s (#20)	36.52s (#24)	23.05s (#20)	22.97s (#15)
Claude haiku 4.5	43.07s (#18)	43.62s (#17)	46.52s (#19)	46.08s (#18)	60.58s (#22)	36.10s (#15)	51.84s (#22)	37.15s (#15)	36.28s (#23)	23.39s (#21)	50.41s (#28)
Gpt oss 20b	42.87s (#17)	43.70s (#18)	43.95s (#18)	49.20s (#20)	60.11s (#21)	48.82s (#21)	37.08s (#14)	49.67s (#18)	32.33s (#20)	22.81s (#19)	30.26s (#21)
Llama 4 Maverick	41.27s (#16)	44.09s (#19)	43.68s (#17)	36.73s (#13)	44.50s (#16)	42.22s (#19)	45.37s (#18)	34.32s (#12)	42.21s (#26)	36.31s (#26)	45.09s (#26)
Claude Opus 4.6	37.87s (#15)	42.32s (#16)	54.46s (#22)	46.71s (#19)	42.76s (#15)	38.19s (#16)	38.72s (#15)	44.29s (#17)	29.02s (#17)	17.29s (#13)	21.89s (#14)
Grok 4.1 fast	36.20s (#14)	36.46s (#14)	39.22s (#13)	40.12s (#15)	31.15s (#11)	35.65s (#14)	64.63s (#25)	41.20s (#16)	30.46s (#18)	21.32s (#18)	29.43s (#18)
Grok 4.20	33.09s (#13)	32.96s (#12)	46.91s (#20)	36.79s (#14)	35.31s (#14)	31.72s (#13)	36.91s (#13)	35.66s (#14)	35.66s (#22)	19.61s (#17)	23.81s (#16)
MiniMax M2.7	27.07s (#12)	34.30s (#13)	26.19s (#10)	29.26s (#12)	34.73s (#13)	29.82s (#11)	30.47s (#11)	31.82s (#11)	18.05s (#10)	14.84s (#11)	14.58s (#9)
Gemini 3.1 Pro Preview	25.92s (#11)	26.14s (#10)	28.82s (#11)	25.29s (#9)	27.89s (#9)	26.67s (#9)	27.39s (#9)	34.50s (#13)	23.25s (#14)	16.62s (#12)	20.46s (#13)
Mimo V2 Pro	25.89s (#10)	26.95s (#11)	29.19s (#12)	26.76s (#11)	32.30s (#12)	27.68s (#10)	28.31s (#10)	30.75s (#10)	18.88s (#11)	17.52s (#14)	15.35s (#10)
Gemini 3.1 Flash Lite Preview	23.17s (#9)	25.97s (#9)	15.68s (#7)	18.29s (#7)	29.88s (#10)	31.61s (#12)	36.07s (#12)	29.29s (#9)	13.92s (#8)	12.31s (#8)	17.37s (#12)
Claude Opus 4.7	20.85s (#8)	23.46s (#8)	23.48s (#9)	26.62s (#10)	26.46s (#8)	24.31s (#8)	21.90s (#7)	27.63s (#8)	11.34s (#7)	9.47s (#7)	11.32s (#8)
Gpt oss 120b	18.03s (#7)	18.21s (#6)	12.63s (#4)	20.54s (#8)	25.83s (#7)	20.13s (#6)	22.44s (#8)	20.26s (#7)	14.20s (#9)	14.53s (#10)	6.51s (#4)
Qwen3.5 35B A3B	14.33s (#6)	18.61s (#7)	17.74s (#8)	17.04s (#6)	17.16s (#6)	14.09s (#4)	16.21s (#6)	18.54s (#4)	7.00s (#4)	6.60s (#4)	8.44s (#7)
Qwen3.5 122B A10B	13.14s (#5)	15.39s (#4)	14.56s (#6)	13.67s (#3)	13.48s (#4)	13.44s (#2)	14.05s (#4)	19.84s (#6)	8.31s (#5)	8.10s (#5)	8.42s (#6)
Gemini 3 Flash Preview	12.51s (#4)	15.88s (#5)	13.77s (#5)	13.93s (#5)	12.99s (#3)	14.36s (#5)	11.85s (#3)	13.66s (#3)	10.30s (#6)	8.69s (#6)	7.81s (#5)
Gemma 4 26B A4B IT	12.38s (#3)	13.71s (#3)	11.14s (#2)	13.89s (#4)	15.14s (#5)	13.88s (#3)	14.92s (#5)	19.48s (#5)	6.38s (#3)	5.69s (#3)	5.48s (#1)
Mistral Small 4	10.56s (#2)	10.48s (#1)	9.58s (#1)	11.41s (#1)	9.21s (#1)	22.29s (#7)	9.79s (#2)	12.12s (#1)	4.88s (#1)	4.38s (#2)	6.49s (#3)
Mistral large 2512	9.27s (#1)	11.02s (#2)	11.39s (#3)	12.33s (#2)	9.96s (#2)	9.45s (#1)	9.74s (#1)	12.16s (#2)	5.79s (#2)	4.09s (#1)	5.78s (#2)