AI cache performance should be measured inside its trust boundary

Cloudflare’s October 26, 2026 disclosure reports cross-request exposure affecting its GLM-5.3 deployment from August 27 to September 9. It says no other Workers AI model was affected. Company report.

This is a specific incident, not evidence that every shared inference service leaks information. It nevertheless raises a broader procurement question: when a supplier reports faster inference, what boundaries were preserved during the measurement? Speed is useful only if the service still performs the work the customer authorized.

The workload includes the separation between customers

Imagine a hypothetical document assistant used by two companies. Both ask it to summarize a contract, and both use the same public instructions. Their private documents remain different. A performance test that combines these jobs into one convenient pool could make the service look attractive while failing to represent the customers’ actual confidentiality requirements.

Our analytical proposal is to treat the permitted sharing group as part of the workload definition. A benchmark should state whether work belongs to one user, one team, one organization or unrelated organizations. Those descriptions are not interchangeable. Even inside a company, a legal investigation and an ordinary support conversation may require different access boundaries.

This changes how a buyer reads a comparison. Suppose one test allows broad reuse and another keeps departments separate. A lower latency in the first test does not, by itself, establish a better implementation. It may reflect a different service being offered. The buyer should first align the permitted sharing scope, then compare performance within that scope.

The argument is not that sharing is inherently wrong. Reusing a public instruction that all participants are entitled to see can be a legitimate design choice. The difficulty is turning a convenient technical grouping into an unexamined authorization decision. A common model, a common billing account or similar wording does not settle who should share derived state.

The customer is also buying more than one answer. The same boundary should hold during busy periods, after jobs end and when usage changes. Separation should therefore sit alongside task type, input size and acceptance criteria as a premise of a performance report, rather than appear as a footnote.

A successful answer is not the whole acceptance test

SGLang’s October 26 disclosure describes a rejected request continuing into a disaggregated handoff and reading unpopulated cache pages. It discusses related lifetime safeguards separately. Maintainer report.

The distinction matters for our analysis: intentional computational reuse and accidental access to another request’s state are different problems. A policy governing the former cannot be assumed to prevent the latter. A buyer needs evidence about both the allowed arrangement and whether the implementation preserves it when work does not proceed normally.

Consider a controlled, fictional acceptance exercise using invented documents rather than customer records. One test group receives a distinctive project label; another has no access to it. The evaluator examines whether any answer carries information from the wrong group, while also testing whether legitimate summaries remain useful. This is a proposal for authorized testing, not a claim about how often any production service fails.

The exercise should include denied, interrupted and retried work because those are part of the service a customer experiences. A test suite containing only completed, well-formed jobs leaves a gap between the published result and the operational question. At the same time, passing such tests does not prove the absence of every leak. It establishes evidence for a specified configuration, scenario and observation period.

A useful report would therefore distinguish three outcomes: an authorized useful answer, an explicit refusal or failure, and an answer that should never have been produced from the available authority. Averaging all returned responses into one success rate would obscure the most consequential difference. Graceful failure can be preferable to a fluent answer whose provenance is wrong.

This distinction also shapes optimization incentives. A target focused only on reducing errors can encourage more responses; a target focused on authorized useful results can make terminating some jobs the correct outcome. This is an argument about acceptance criteria, not an allegation about any supplier’s incentives.

Compare efficiency after fixing the boundary

As a separate design example, vLLM documents optional request salts that restrict prefix-cache reuse to matching salts. This is not a remedy for every cache-lifetime defect. vLLM documentation.

The procurement implication is not to mandate that particular implementation. It is to ask for a sharing policy that the buyer can describe and the operator can test. An application owner might allow reuse within a project while separating unrelated clients. Another might accept a slower service to reduce the amount of shared request-derived state. Neither choice can be evaluated from tokens per second alone.

There is a reasonable objection: demanding increasingly narrow separation can reduce the economic benefit of a common platform, while elaborate tests delay adoption. That trade-off should be visible rather than hidden. The appropriate comparison is between designs meeting the same required boundary, including their latency, capacity and operating burden. It is not between an isolated service and an unconstrained benchmark presented as equivalent.

The same principle applies when a team later changes its access model. Joining two workspaces may create new sharing opportunities; removing a participant may narrow them. In our proposed governance model, those changes should trigger a review of the performance assumptions, not merely an edit to a user directory. A fast configuration is valuable only while its permissions remain suitable for the work.

The disclosures do not establish which architecture is best for every customer. They support a more careful question for evaluation: how much useful throughput remains after the required separation has been made explicit and tested? That is a more meaningful basis for choosing an inference service than a speed figure detached from its trust boundaries.