Multilingual AI Needs a Complete Case File

On October 15, 2026, Hugging Face described BERT tokenization speedups at pull-request stage. Faster encoding does not itself reduce token counts. Report

That is a useful starting point for a different product question: what survives the journey from a customer's message to an AI-assisted decision? Open Market Notes proposes testing the completeness of the case presented to the model. This is an original evaluation design, not a reported product feature or a benchmark we have run.

Begin with the decision

Imagine a fictional appliance-repair service operating in several languages. A case contains a device identifier, an appointment request, and a later authorization to replace the device instead of repairing it. Success means recognizing the replacement authorization and routing the case accordingly. A fluent appointment confirmation would be wrong even if it arrived quickly.

Create equivalent versions of that case with bilingual reviewers. Preserve the sequence of events, the authority behind the exception, and the requested action. The texts need not have the same number of characters or sentences. Their decision-relevant content must match. Include naturally mixed-language cases too, rather than assuming every customer writes in one language from beginning to end.

The reference answer should identify which records justify the decision. It should also specify when the assistant must ask for clarification. If the authorization is unsigned or refers to another device, escalation can be the correct outcome. A test that rewards decisive answers regardless of evidence would measure the wrong behavior.

A 2023 study found that translations can produce unequal token counts. Paper

Inspect each boundary

In our proposed test, keep a record at each handoff: the original case, the text selected for retrieval, the assembled model input, and the final action proposal. Check where the replacement authorization first disappears or changes meaning. This separates an answer-generation error from a system that never supplied the decisive record.

Run matched versions with the exception near the beginning, middle, and end. Then add irrelevant but realistic conversation around it. These are controlled variations of one fictional case, not claims about any language's users. The purpose is to discover whether order or surrounding material changes the available evidence before the model decides.

Compare a fixed-token input rule with a rule that reserves room for the required records. Keep the model and scoring criteria constant. The second rule is not automatically better: it could select the wrong records, admit stale instructions, or consume more resources without improving decisions. The comparison must retain those failures rather than assuming that a larger allowance solves the problem.

The October LCT preprint explores separating linguistic-structure discovery from vocabulary construction. Preprint

The fixed-model comparison above concerns which records are supplied. It excludes vocabulary changes: those belong in a separate experiment with a suitably trained or adapted model, not a drop-in swap into the unchanged model. Record selection, case translation and history reduction should each be tested separately before evaluating their combined effect.

Count reopened cases

An operational scorecard could record whether the decisive evidence reached the model, whether the proposed action matched that evidence, and whether a reviewer had to reopen the case. Track latency and resource use beside those outcomes. A fast response that creates another contact may simply move work elsewhere.

Report results by language and case type, alongside the overall result. Keep the sample sizes visible. Do not turn a handful of unusual examples into a ranking of entire languages, and do not let a large easy category conceal a smaller category with missing evidence. The test should reveal where further investigation is needed.

The original case should remain available to reviewers when a translated or shortened version is used. A compact summary can be useful, but the test must check whether it preserved the authorization's scope and timing. “Replacement approved” and “replacement requested” lead to different actions in this fictional workflow.

A service promise, not a token promise

The proposed goal is consistent access to decision-critical evidence, not identical computational treatment or a guarantee of fairness. Some cases will require more work; some will remain unresolved. Those differences should be visible rather than hidden inside a single throughput average.

The useful commercial question is whether the same valid request receives the same defensible handling across languages. Token efficiency can contribute to that goal. A complete case file, a clear acceptance rule, and an honest record of unresolved cases make it possible to test whether it actually does.