Safety reports need to show how cases move through the queue

On October 30, 2026, OpenAI announced a Community Safety Report planned for early 2027 and safeguards due to begin rolling out in coming weeks. These remain commitments, rather than a completed evaluation. Announcement

The announcement raises a practical question for AI governance: what should readers be able to infer from a future safety report? A total of interventions can describe activity. Understanding whether the response system is adequate requires information about the cases it receives, the time it needs and the limits of the outcomes it can observe. The framework below is Open Market Notes' analysis, not a description of OpenAI's internal operations.

Define the unit before comparing totals

A single person may generate several signals, several conversations and several review decisions. If a report counts all of these as cases, growth can reflect repeated handling of the same concern. Counting only accounts has the opposite weakness: a long-running case can disappear into one unit even as it consumes substantially more attention.

A useful reporting design would distinguish signals, reviewed cases and actions, while explaining when related records are grouped. This need not expose individual identities. It requires a stable counting rule. When that rule changes, a comparison should show the effect of the change separately from any change in the underlying workload.

The denominator also needs a purpose. Actions per active user could describe operational scale, but would say little about the share of serious concerns handled correctly. Actions per reviewed case answers another question and may change simply because the criteria for entering review changed. Neither ratio should be presented as a universal safety score.

Follow a cohort through the queue

Consider a hypothetical service receiving a batch of cases near the end of a reporting period. Its team completes many older cases during the same week. A dashboard showing arrivals and completions can look balanced even while the new batch waits. Comparing totals without case age obscures this distinction.

A cohort view would track cases entering during the same interval, recording what fraction is still awaiting a decision after relevant elapsed times. Urgency categories should remain separate. A small number of time-sensitive cases can matter more than a large, quickly resolved low-priority queue. A single average response time could hide precisely the part of the distribution that management needs to understand.

Resource planning then becomes more concrete. If additional detection brings more cases into review, the organization should examine whether specialist availability, handoffs and decision authority can absorb them. Increasing detection while leaving a bottleneck unchanged may lengthen waits. That is an operational possibility to test, not evidence that it is happening at any named company.

Include the cost of correcting mistakes

An action can be appropriate on the information initially available and later require revision. Reports that count only the initial decision miss the work of reviewing appeals, restoring access where appropriate and correcting the record. Conversely, an overturned decision does not by itself establish that the original reviewer acted negligently.

A more informative account would explain which decisions can be revisited, how long reconsideration takes and what information caused a change. Changes in policy should be separated from corrections under an unchanged policy. Otherwise, a rising reversal rate could be misread as worse performance when the organization has actually broadened review access or improved error detection.

NIST's AI RMF includes monitoring, response times and feedback in risk evaluation. It provides a useful reference point; it does not certify a provider's results. The specific reporting design proposed here would still need testing against its intended use.

Keep observed outcomes separate from causal claims

A completed referral or intervention is a documented action. Establishing what harm it prevented requires a much harder comparison with what would otherwise have happened. That counterfactual may remain unavailable. External agencies may also be unable to return complete case outcomes, and delayed information can leave recent cohorts less mature than older ones.

A credible report can acknowledge those limits without becoming uninformative. It can distinguish completed actions, externally confirmed follow-up and outcomes that remain unknown. It can state the observation window and revise earlier cohorts when additional evidence arrives. Missing outcomes should remain visibly missing rather than being treated as either success or failure.

Privacy places another constraint on disclosure. Publishing detailed small groups could expose people or sensitive processes. Aggregated ranges, suppression of small cells and a clear explanation of what cannot be reported may be preferable to an apparently precise but unsafe table. Such choices also limit comparisons and should be explained alongside them.

For readers assessing the forthcoming report, the useful question is whether the evidence supports a decision: where more review capacity is needed, which steps should change and which conclusions remain premature. A report with fewer claims and a clear account of unresolved cases can be more decision-useful than a larger collection of intervention totals.