Beyond Accuracy: Decomposing the Reasoning Efficiency of LLMs
Earlier version: NeurIPS 2025 Workshop on Efficient Reasoning ↗
Accuracy per token hides why a model is efficient. An exact two-layer decomposition separates answer availability, correctness and cost, and shows that relative cost changes with workload and can reverse.
Abstract
Accuracy per token summarizes reasoning efficiency but does not reveal whether differences arise from failed attempts, costly successful answers, or disproportionately long reasoning relative to the effort a problem demands. We introduce a two-layer protocol that connects these distinctions to the same efficiency score. The outcome layer separates answer availability, correctness among available answers, and generation cost. The workload layer relates cost to task structure, separating mean generated tokens per workload unit from their association with workload. Across 785,541 responses on twelve benchmarks, cost gaps persist among successful answers: Qwen3.5-27B generates 1.79–3.46 times as many tokens as Gemma-4-31B for correct outputs on shared problems across ten benchmarks. Relative costs also change with workload. On ZebraLogic, Qwen3.5-27B’s cost relative to GPT-OSS-120B changes from 4.49 at low workload to 0.78 at high workload, despite similarly long responses on average. Across nine checkpoints on four annotated benchmarks, simultaneous 95% confidence intervals support relative-cost changes in 82 of 144 model-pair comparisons and effort-induced changes in 21 of 36 within-checkpoint comparisons. These results show that successful-answer cost and workload response are complementary properties of reasoning models: cost gaps can persist on shared successful questions while changing substantially with task structure and inference settings.
Key numbers
- 785,541
- responses across twelve benchmarks
- 1.79–3.46×
- tokens Qwen3.5-27B spends relative to Gemma-4-31B on shared correct answers, across ten benchmarks
- 4.49× → 0.78×
- Qwen3.5-27B’s cost relative to GPT-OSS-120B on ZebraLogic, from low to high workload
- 82 / 144
- model-pair comparisons with a workload-dependent change in relative cost, under simultaneous 95% intervals
- 21 / 36
- within-checkpoint comparisons where raising reasoning effort changed the workload response
Interactive
Decompose a gap
Correct answers per thousand tokens is one number per model. Behind it sit three. How often an answer is available at all, how often an available answer is correct and how many tokens a response costs. For two models the log ratio of their scores is exactly the sum of three log ratios. Move the sliders and see which term carries the gap.
The outcome layer only. The paper’s workload layer splits mean tokens further into mean workload × tokens per workload unit × a coupling term, and the comparisons on shared questions and workload groups are where the reversals show up.
Cite
@article{kaiser2026beyond,
title = {Beyond Accuracy: Decomposing the Reasoning Efficiency of LLMs},
author = {Kaiser, Daniel and Frigessi, Arnoldo and Ramezani-Kebrya, Ali and Ricaud, Benjamin},
journal = {arXiv preprint arXiv:2602.09805},
year = {2026}
}