Router Lab
Medium impactLive published free itemLiteLLM AutoRouter v2 claims 45% lower cost per solved task
A classifier that estimates each tier's odds of success and routes to the cheapest tier clearing a probability threshold. Benchmark claims 27% more tasks solved at 45% lower cost per solve.
What changed
LiteLLM shipped AutoRouter Heuristic v2. It estimates each model tier's probability of solving a task and routes to the cheapest tier above a threshold, with no per-request classifier call on the critical path and pretrained success-probability tables available before first use. On a 21-task benchmark subset it reports 14 of 21 solved against 11 of 21 for v1, $0.70 per solve against $1.28, and about 30% lower total spend across the run.
Why it matters
Routing to the cheapest model that can finish the job is the correct instinct, and cost per solved task is the right metric rather than cost per call. The caveat is the benchmark: 21 tasks is a small sample published by the vendor, so treat the percentages as a reason to test rather than a result to quote.
Take
A classifier that estimates each tier's odds of success and routes to the cheapest tier clearing a probability threshold. Benchmark claims 27% more tasks solved at 45% lower cost per solve.
What to do
Do not adopt the published figures. Take twenty tasks that represent your real work, run them through your current setup and through the router, and compare cost per accepted result. A router that solves more at lower cost on your tasks is worth adopting. One that only does so on the vendor benchmark is not.
Evidence
No public sources are linked yet.