AutoBench Run 5 - December 2025

Latest AutoBench run with models Gpt 5.2, Claude Opus 4.5, Gemini 3 Flash and more

Past

Date

December 19, 2025

Version

2025-12-19

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

AutoBench provides a complete dashboard with highly customizable domain-specific average ranks and precise efficiency metrics (average answer cost, average answer duration, P99 Latency). For further detail also check our Hugging Face Leaderboard.

Model	Score	Avg Cost ($ Cents)	Avg Latency (sec)	P99 Latency (sec)	Iterations
DeepSeek 3.2 Speciale	4.14 (#17)	0.47 (#17)	310s (#37)	833s (#38)	288
Qwen3 235B A22B Thinking 2507	4.20 (#13)	0.32 (#13)	317s (#38)	811s (#37)	283
Gpt 5.2 Pro	4.48 (#1)	81.88 (#38)	261s (#36)	784s (#36)	303
Kimi k2 thinking	4.32 (#6)	1.86 (#26)	248s (#35)	729s (#35)	287
GLM 4.6	4.13 (#18)	1.25 (#24)	187s (#33)	630s (#34)	306
Gpt 5.1	4.38 (#5)	10.80 (#35)	227s (#34)	627s (#33)	310
Grok 4	4.20 (#12)	8.12 (#34)	180s (#32)	562s (#32)	293
Claude sonnet 4.5	4.30 (#8)	11.39 (#36)	170s (#30)	477s (#31)	307
DeepSeek R1 0528	4.12 (#19)	0.99 (#23)	171s (#31)	477s (#30)	308
Minimax m2	3.99 (#26)	0.71 (#20)	137s (#27)	473s (#29)	308
Gpt 5.2	4.43 (#2)	7.36 (#33)	130s (#26)	434s (#28)	312
GLM 4.5 Air	3.86 (#31)	0.54 (#19)	163s (#29)	425s (#27)	306
Deepseek v3.2	4.11 (#20)	0.09 (#7)	125s (#25)	410s (#26)	311
Claude opus 4.5	4.39 (#4)	17.26 (#37)	144s (#28)	373s (#25)	313
Qwen3 235b a22b 2507	3.98 (#27)	0.19 (#10)	105s (#22)	337s (#24)	302
Kimi K2 0905	4.11 (#21)	0.33 (#14)	83s (#17)	329s (#23)	312
Claude haiku 4.5	4.17 (#16)	3.79 (#29)	111s (#23)	317s (#22)	312
Gpt oss 120b	4.18 (#14)	0.11 (#8)	75s (#13)	292s (#21)	292
Olmo 3.1 32b Think	3.85 (#32)	0.00 (#2)	122s (#24)	270s (#20)	307
Gpt 5 nano	4.06 (#22)	0.34 (#15)	100s (#21)	269s (#19)	309
Gpt 5 mini	4.29 (#10)	0.91 (#22)	93s (#20)	258s (#18)	312
Llama 3.3 nemotron super 49b v1.5	3.78 (#34)	0.18 (#9)	76s (#15)	240s (#17)	311
Qwen3 next 80b a3b thinking	4.03 (#24)	0.75 (#21)	78s (#16)	227s (#16)	312
Gemini 2.5 pro	4.29 (#9)	6.48 (#31)	87s (#18)	222s (#15)	313
Nemotron nano 9b v2	3.50 (#37)	0.08 (#5)	67s (#11)	212s (#14)	311
Grok 4.1 fast thinking	4.21 (#11)	0.27 (#12)	69s (#12)	207s (#13)	306
Mistral large 2512	3.94 (#29)	0.51 (#18)	90s (#19)	198s (#12)	307
Gemini 3 pro preview	4.41 (#3)	6.85 (#32)	76s (#14)	186s (#11)	312
Gpt oss 20b	3.78 (#35)	0.07 (#4)	39s (#5)	183s (#10)	310
Gemini 2.5 flash	4.17 (#15)	2.12 (#28)	66s (#10)	174s (#9)	312
Ministral 8b 2512	3.57 (#36)	0.05 (#3)	31s (#4)	154s (#8)	306
Mistral medium 3.1	3.81 (#33)	0.38 (#16)	52s (#8)	147s (#7)	306
Gemini 3 Flash Preview	4.30 (#7)	1.95 (#27)	46s (#6)	137s (#6)	313
Nova Premier v1	3.47 (#38)	1.30 (#25)	52s (#7)	135s (#5)	312
Nova 2 lite v1	4.06 (#23)	3.94 (#30)	61s (#9)	132s (#4)	277
Nemotron 3 Nano 30B A3B	4.03 (#25)	0.00 (#1)	30s (#3)	98s (#3)	314
Gemini 2.5 flash lite	3.95 (#28)	0.21 (#11)	20s (#1)	69s (#2)	313
Grok 4.1 fast	3.88 (#30)	0.08 (#6)	24s (#2)	57s (#1)	312