The price of a higher math score is not the price of a better AI workflow
MathArena’s new results reveal a capability–cost trade-off, but model selection still depends on which errors matter and how they are checked.

MathArena added Carbon and Fable results on November 13, 2026. Aggregate scores mix observations and estimates: an evaluation update, not a new model launch today. Source and methodology
The model pages currently list expected performance and expected cost per problem of 85.4% and $15.70 for Gemini 4 Carbon (high), 82.8% and $40.20 for Claude Fable 5.2 (max), and 80.6% and $1.07 for GPT-6.1 Sol (max). These are benchmark estimates, not quotations for a customer's workload or measurements of physical compute.
The following is Open Market Notes analysis. The interesting question is not simply which number is highest. It is when an additional chance of solving a difficult problem is worth paying for, and whether a team can identify those occasions before paying.
A leaderboard is not an invoice
A research team with one difficult proof and an application handling thousands of routine requests face different decisions. The former might value a single new line of reasoning far above its inference bill. The latter may need predictable latency, a stable budget and a clear process for unresolved requests. A price–performance graph can help both teams choose experiments. It cannot choose their objective for them.
A score is also a property of a test configuration. Changing the time allowed, available tools, retries or stopping rule creates a different experiment, even if the model name stays the same. Comparing a maximum-effort result with a cheaper production configuration without retesting would conceal the very trade-off the buyer is trying to measure. The relevant unit is the complete system submitted to evaluation, not a brand name detached from its operating conditions.
This distinction matters before making a cost ratio. Dividing one headline cost by another gives a descriptive ratio for those estimates. It does not prove that one vendor requires that many times more computation, nor that its price is that many times higher for every task. Token prices, response lengths, work attempted and execution policy can all matter. The public numbers do not isolate their causal contributions.
An average cannot tell you which failure to buy away
MathArena's May 2026 platform paper describes evaluation extending beyond final answers to proofs, research mathematics and Lean formalization. Those tasks test different aspects of mathematical work. Research paper
Consider two hypothetical systems with identical average scores. One might fail on the same narrow class of problems every time; the other might make dispersed, intermittent mistakes. A tutoring service, a theorem-proving assistant and a verification pipeline could reasonably rank them differently. The average alone cannot reveal whether a particular failure is easy to detect, expensive to repair or likely to be repeated.
Nor should a gap between aggregate scores be read as a measured improvement on each question. Statistical aggregation can make incomplete evaluations easier to compare. The underlying inference still needs assumptions. Research on combining heterogeneous benchmarks makes precisely this comparability problem explicit; it does not turn a predicted result into an observed answer. Benchmark-combination research
For procurement, the useful next step is a paired test: run the same held-out problems through each candidate under recorded constraints, then inspect disagreements. Which problems does the more expensive system solve that the cheaper one cannot? Are those problems common in the intended product? Can an evaluator reliably recognize the difference? These questions turn a general ranking into evidence about a specific decision.
It is also important not to confuse easy-to-score work with the most valuable work. If a product mainly handles ambiguous questions or incomplete context, a definite mathematical answer supplies only part of the evidence. A test set should represent real needs, while including challenging cases with transparent weights, rather than replacing the task distribution to emphasize a favored strength.
A cheaper first attempt only helps if escalation works
One possible design is to begin with a lower-cost system and reserve a more expensive one for unresolved cases. This is a hypothesis to test, not a saving demonstrated by the leaderboard. If the first system confidently produces a wrong answer and the routing layer accepts it, no escalation occurs. If the routing layer sends almost everything to the second system, the first attempt merely adds cost and delay.
The routing decision therefore deserves its own evaluation. A mathematically valid certificate can provide stronger evidence than a model's confidence statement, but checking a formal object is not the same as proving that the right real-world question was encoded. An automated checker can verify the wrong specification perfectly. Human review may still be needed at the boundary between the user's intention and the formal task.
The accounting should include first attempts, escalations, retries, checking and rework. A useful denominator is accepted, independently checked work, with latency and remaining error reported alongside it. This does not mean every application needs formal verification. It means the acceptance criterion must be stated before comparing costs, rather than changed after a preferred model wins.
The accounting should also admit a less attractive result: some tasks cannot yet be completed reliably within the budget. Allowing abstention or human escalation may reduce apparent automation coverage, but avoids counting undetected failures as successful deliveries. That is more useful for long-term budgeting and for reevaluation after model updates.
Keep the decision reversible
The new results justify revisiting evaluations, not rewriting an entire product around a headline rank. A team can retain a representative test set, record its prompts and stopping rules, and preserve failed outputs for later diagnosis. Difficult cases should be included deliberately, but weighted transparently rather than allowed to replace the normal workload.
The strongest interpretation of this update is consequently conditional: higher measured capability may be valuable where difficult failures are costly, while a lower-cost configuration may be preferable where its outputs can be checked and its limitations managed. Neither conclusion establishes a universal winner. A well-designed evaluation should tell a team not only which system to use today, but which change in workload, error tolerance or price would make that choice no longer sensible.
Information only. No investment, legal, tax, or financial advice.