Who gets to wait when AI capacity is shared?

On October 20, 2026, llm-d released version 1.0, emphasizing multi-tenant operations. The same day, Red Hat described priority-aware serving in Red Hat AI 3.5. These announcements raise a question that extends beyond infrastructure: when several products share inference capacity, whose work may wait, and what has the service promised to everyone else?

The analysis below concerns that product-design question. It does not independently validate either project's performance claims or offer a deployment recipe. Sharing a resource creates a need to allocate scarce time. A queue may encode that allocation, but the meaning of its labels must come from the organization operating it.

Waiting has different consequences

Consider a hypothetical company running three tasks: an assistant helping a staff member prepare for an imminent meeting, a document-indexing job needed for tomorrow's search, and a background agent generating optional suggestions. None is a reported customer deployment. The example illustrates why dividing work into “interactive” and “batch” leaves an important decision unresolved.

The meeting assistant has value only while its answer can still influence the conversation. Indexing can tolerate a delay now, yet becomes important as tomorrow's opening time approaches. The optional agent may respond quickly and still be expendable. Speed requirements, business importance and the latest useful completion time are separate properties. Google's SRE discussion of overload likewise distinguishes request criticality from latency requirements. That is background engineering guidance, not evidence that any particular inference product implements a deadline policy.

Our proposed starting point is therefore a loss-of-delay map. Ask what becomes worse after a request waits, who experiences that loss, and when the output ceases to be useful. An assistant whose user has left the page should not keep its urgency merely because its request originally came from an interactive interface. Conversely, an overnight task should not remain permanently subordinate when its promised delivery window is about to close.

Acceptance needs a meaning

For the hypothetical meeting assistant, imagine an answer is needed within thirty seconds. That number is an illustrative product requirement, not a measured performance result. If the request spends most of that interval waiting, starting generation just before the deadline may produce an operationally successful response that arrives after the decision. A count of completed requests would conceal the failure.

A useful service contract would separate “received,” “accepted for execution,” and “completed while still useful.” It would specify whether expired work is cancelled, whether partial output is usable, and whether a caller may choose a later delivery instead. Those distinctions matter before an operator decides how many priority classes to configure. Otherwise the platform and the application can disagree about whether a slow success satisfied the promise.

Priority alone cannot settle these questions. Red Hat explicitly notes limitations from sustained high-priority traffic and long outputs already admitted. A business deadline also includes time outside the inference queue: retrieving documents, obtaining a tool result, and returning the answer. Assigning the model call the highest label does not reserve time for those other steps. Any credible completion promise needs an end-to-end budget and evidence from the actual workload.

Who may declare an emergency?

Priority labels introduce an incentive problem. If higher priority is available at no organizational cost, every team has a reason to claim it. The label then describes how forcefully a team negotiates rather than the consequences of delay. A technically correct scheduler can faithfully implement an unhelpful policy.

One defensible approach would give service owners responsibility for the classification, with a recorded reason for exceptions and a review of unusually persistent urgent traffic. In the hypothetical company, a meeting assistant could have bounded urgent access, while the indexing owner could request a time-limited escalation as its deadline approaches. This is an editorial proposal, not a description of capabilities supplied by the release. Its purpose is to make priority a limited promise that someone is accountable for granting.

Agent workflows make the boundary more important. Suppose one user request launches several optional branches. Automatically treating each branch as equally urgent could let one customer interaction consume the allocation intended for many interactions. The application should explain which branches are essential, which can be cancelled, and how much concurrent work the whole task may create. Measuring fairness at the request level alone would miss that relationship between a parent task and its descendants.

There is also a legitimate case for keeping some capacity separate. Isolation can make a narrow, important service easier to reason about, although it reduces the flexibility of a shared pool. The choice depends on the promise being sold or relied upon. Consolidation should be justified against that promise rather than treated as an objective that every workload must satisfy.

Test the promise, including the people who wait

A useful evaluation would reproduce the hypothetical company's ordinary workload, then hold a demand surge long enough to observe the deferred work. Brief bursts might show that urgent answers stay responsive while hiding an indexing job that never catches up. The evaluation should ask whether each class finishes within its own useful window, not merely whether the aggregate response time improves.

It should also distinguish deliberate postponement from abandonment. How much work was accepted and later cancelled? How long did the oldest deferred task wait? Did optional agent branches crowd out independent users? Which missed deadlines were caused upstream or downstream of inference? These are proposed operational questions; they are not results reported by the vendors.

Some uncertainty will remain. Future request lengths, simultaneous peaks and the usefulness of partial answers are not perfectly knowable in advance. A platform can respond by narrowing its promises, documenting what happens under sustained contention, and giving applications an explicit way to handle work it cannot complete in time. The practical significance of priority-aware serving lies in making allocation configurable. Whether that allocation constitutes a trustworthy service depends on a less automatic decision: defining, and testing, what each waiting user is entitled to expect.