Anti-Scam AI Needs an Attention Budget

Gemma’s anti-scam rollout remains planned. The useful question for the next stage is how much attention a protection system asks people to spend, and what they receive for it. This is our proposed evaluation framework, not a report of measured model performance.

Start with the denominator

Consider a hypothetical test containing 10,000 conversations, of which 100 are scams. A detector that catches 90% of scams and wrongly flags 1% of legitimate conversations produces 90 correct warnings and 99 false warnings. Only about 47.6% of its 189 warnings identify a scam. Ten scams still pass through.

Those numbers are illustrative, not estimates for Gemma or any deployed service. They show why a strong detection rate does not, by itself, tell a user how much confidence to place in an alert. The proportion of scams in the tested population matters. A collection of already-suspicious messages and an ordinary inbox can produce very different experiences even with identical error rates.

This creates an attention budget. Every unnecessary interruption consumes time and may make the next warning easier to ignore. Conversely, suppressing too many warnings can leave harmful conversations unchallenged. A credible evaluation needs to show both costs rather than selecting whichever metric looks better.

Separate the decisions

Existing ScamShield documentation supplies a useful, separate reference point. Its checking feature works without enabling call-blocking or SMS-filtering permissions. Its reporting guidance describes automatic reports of senders associated with high-probability scams or spam/promotional content. These documents describe existing ScamShield functions; they do not establish the implementation of any separate model integration.

The analytical lesson is that checking, warning, reporting and blocking deserve separate decision rules. A person might welcome a private prompt to pause yet want a clearer explanation before information is submitted or a sender becomes unreachable. Combining these steps under a single protection switch would make both evaluation and consent harder to interpret.

A product test should therefore ask four questions: what was inspected, what was displayed, what information left the device, and what action changed the conversation? Each answer needs its own record. A warning counter cannot answer all four.

Measure the intervention

The first useful scorecard would report correct warnings, false warnings and missed scams against explicitly defined denominators. Results should be broken out by language and scenario, with uncertainty shown where samples are small. Evaluators should also define when a long conversation counts as one event, so repeated alerts do not inflate apparent protection.

The second scorecard would follow what happens after an alert. Did the person understand it? Did they pause long enough to verify independently? Could they dismiss an incorrect warning and continue a legitimate exchange? These are proposed tests, not documented capabilities or known shortcomings of the announced model.

Most importantly, a detected scam and a prevented loss are different outcomes. Attributing avoided losses requires evidence about what would otherwise have happened. Alert volume alone cannot supply that counterfactual. Nor should a quiet device be presented as proof that every conversation is safe.

Keep the evidence proportionate

Evaluation should not require collecting every private conversation. A bounded test programme could combine consented review samples, clearly labelled synthetic scenarios and privacy-preserving aggregate measures. Its limitations should remain visible: a laboratory exercise cannot establish real-world loss reduction on its own.

The near-term milestone worth watching is a published account of how an intervention earns user attention: its operating population, error trade-offs, action boundaries and method of measuring benefit. That would make the next deployment announcement substantially more informative than a larger count of messages scanned.