AutoBench Run 6 - August 2026

Twenty models scored on 252 questions across ten subject domains. Score, cost, and latency are the per-answer committee rank, billed cost, and generation time. Routers in the AutoBench Router Benchmark are scored against this answer matrix.

Latest

Date

August 29, 2026

Version

2026-08-29

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

Evaluate model performance across 10 specialized knowledge domains with normalized scores. Identify which models excel in specific areas—whether you need strong coding capabilities, creative writing prowess, or expertise in science and mathematics. Average scores reflect overall domain competency.

Model	Average (All Topics)	Computer Science	Philosophy	Social Science	Language	Science	Technology	Arts	Literature	History	Agentic
Nemotron 3.5 Lightning	4.36 (#16)	4.07 (#16)	4.6 (#19)	4.59 (#17)	4.22 (#16)	4.3 (#17)	4.25 (#20)	4.1 (#16)	4.46 (#16)	4.51 (#17)	4.51 (#11)
Qwen3.5-9B	4.15 (#19)	3.44 (#20)	4.62 (#18)	4.41 (#19)	4.15 (#18)	4.51 (#15)	4.26 (#19)	3.54 (#20)	4.08 (#19)	4.05 (#20)	4.47 (#12)
Grok 4.3	4.44 (#15)	4.27 (#15)	4.7 (#14)	4.75 (#12)	4.44 (#15)	4.41 (#16)	4.42 (#17)	4.08 (#17)	4.48 (#15)	4.64 (#16)	4.31 (#17)
Muse Spark 1.2	4.56 (#13)	4.43 (#11)	4.77 (#9)	4.69 (#15)	4.64 (#10)	4.65 (#12)	4.42 (#18)	4.36 (#11)	4.68 (#8)	4.71 (#9)	4.31 (#16)
Nemotron 3 Ultra	4.49 (#14)	4.34 (#13)	4.74 (#11)	4.73 (#14)	4.49 (#14)	4.74 (#8)	4.48 (#16)	4.11 (#15)	4.62 (#11)	4.67 (#13)	4.17 (#19)
Gpt oss 120b	4.12 (#20)	3.97 (#19)	4.34 (#20)	4.32 (#20)	4.07 (#20)	3.96 (#20)	4.5 (#15)	3.64 (#19)	4.05 (#20)	4.06 (#19)	4.45 (#14)
Grok 4.6	4.59 (#11)	4.62 (#4)	4.78 (#8)	4.76 (#10)	4.68 (#5)	4.67 (#11)	4.54 (#14)	4.4 (#10)	4.66 (#10)	4.73 (#5)	4.24 (#18)
Gemma 4 31B IT	4.35 (#18)	4 (#18)	4.63 (#17)	4.52 (#18)	4.08 (#19)	4.22 (#18)	4.56 (#13)	4.32 (#12)	4.55 (#14)	4.65 (#15)	4.16 (#20)
MiniMax M3	4.65 (#8)	4.52 (#6)	4.75 (#10)	4.78 (#6)	4.56 (#13)	4.7 (#9)	4.58 (#12)	4.43 (#9)	4.73 (#5)	4.7 (#10)	4.76 (#6)
Qwen3.8 Max	4.74 (#2)	4.65 (#3)	4.82 (#4)	4.8 (#4)	4.67 (#6)	4.86 (#1)	4.6 (#11)	4.59 (#3)	4.76 (#2)	4.79 (#3)	4.82 (#2)
Muse Glimmer 30B	4.35 (#17)	4.06 (#17)	4.73 (#12)	4.63 (#16)	4.18 (#17)	4.18 (#19)	4.61 (#10)	3.87 (#18)	4.33 (#18)	4.45 (#18)	4.64 (#10)
DeepSeek V4 Flash 0731	4.59 (#12)	4.32 (#14)	4.72 (#13)	4.77 (#8)	4.69 (#4)	4.77 (#6)	4.62 (#9)	4.31 (#13)	4.44 (#17)	4.69 (#12)	4.46 (#13)
Hy3	4.61 (#10)	4.45 (#8)	4.66 (#16)	4.76 (#9)	4.65 (#9)	4.53 (#14)	4.64 (#8)	4.25 (#14)	4.61 (#13)	4.71 (#8)	4.76 (#7)
GLM 5.3	4.65 (#9)	4.36 (#12)	4.81 (#5)	4.76 (#11)	4.63 (#12)	4.62 (#13)	4.65 (#7)	4.54 (#6)	4.62 (#12)	4.72 (#7)	4.77 (#4)
GPT-5.6 Luna	4.74 (#3)	4.79 (#1)	4.84 (#3)	4.86 (#1)	4.74 (#3)	4.83 (#3)	4.71 (#6)	4.71 (#1)	4.73 (#4)	4.83 (#2)	4.43 (#15)
GPT-5.6 Sol	4.78 (#1)	4.75 (#2)	4.86 (#2)	4.84 (#2)	4.79 (#1)	4.81 (#4)	4.71 (#5)	4.62 (#2)	4.74 (#3)	4.83 (#1)	4.84 (#1)
Gemini 3.7 Flash	4.68 (#6)	4.45 (#9)	4.81 (#7)	4.78 (#7)	4.65 (#8)	4.77 (#7)	4.73 (#4)	4.45 (#8)	4.72 (#6)	4.75 (#4)	4.66 (#9)
Kimi K2.6	4.69 (#5)	4.45 (#10)	4.81 (#6)	4.8 (#5)	4.66 (#7)	4.84 (#2)	4.75 (#3)	4.55 (#5)	4.67 (#9)	4.7 (#11)	4.69 (#8)
Claude Sonnet 5	4.73 (#4)	4.6 (#5)	4.86 (#1)	4.83 (#3)	4.74 (#2)	4.79 (#5)	4.77 (#2)	4.57 (#4)	4.76 (#1)	4.65 (#14)	4.79 (#3)
GLM 5.2	4.67 (#7)	4.48 (#7)	4.69 (#15)	4.74 (#13)	4.63 (#11)	4.67 (#10)	4.82 (#1)	4.51 (#7)	4.7 (#7)	4.72 (#6)	4.77 (#5)