Enterprise & Security

Enterprise Frontier Safeguards at Uber: an insider threat program for AI agents

3 min read AI-generated

Uber open-sourced a tool called AI Detection and Response. It gathers real-time telemetry inside agent harnesses and is meant to take Anthropic's classifier output as a second data source.

Featured image for "Enterprise Frontier Safeguards at Uber: an insider threat program for AI agents"

Philip Martin is Chief Information Security Officer at Uber, a platform handling more than 40 million trips and delivery orders a day. In an interview Anthropic published on October 13, he makes the case that AI agents need the same thing employees already get: an insider threat program.

What frontier models changed

His team worked with Claude on threat modelling before Claude Code existed, assembling the context by hand. Now Claude gathers that context itself from code, design docs and ticket history. Martin looks further ahead: he imagines every service eventually having its own agent that understands it deeply, with those agents talking to each other about the threat model the way the services talk to each other in production.

Who keeps control of the data

Uber worked with the Enterprise Frontier Safeguards team, and Martin names the tension plainly: Anthropic wants to prevent misuse of its frontier models, while Uber has to protect customer data, trade secrets and proprietary information. The safeguards resolve it by staying in place while Uber’s data remains inaccessible to Anthropic without permission. A safeguard fires at Uber, and what happens next is Uber’s decision. “You put us in the driver’s seat,” he says about the design.

ADR sits beside it, not underneath it

Uber open-sourced a product called AI Detection and Response. It gathers real-time telemetry in the agent harnesses the company deploys and can act in real time. Martin expects Enterprise Frontier Safeguards to feed into ADR as a data source: Uber’s own view of what’s happening, plus the output of Anthropic’s classifier, both inside the same detection environment. Together with Uber’s LLM gateway, which sets the perimeter for AI use inside the company, that’s the architecture.

Where Martin would send defenders next

He names two priorities. First, real-time detection and intervention, as close as possible to the moment a human types a prompt — the nudge that says “you probably don’t want to do that,” or that the tool this model is about to call isn’t one you want called. Second, understanding emerging patterns, because constraints always create new behaviour. That becomes a loop: look at the data, find the patterns, build the safeguard, wait for the next pattern.

The analogy holds up better than it sounds

Martin’s argument is disarmingly simple. Every company runs an insider threat program for its people, logging included, nobody likes it, and nobody argues about it. The reason: humans occasionally act outside the bounds of their role. The same holds for models, minus the intent. What’s interesting is that the same demand showed up twice this week: here from a company already building it, and in the open letter from 400 scientists as a call for standards imposed from outside. The conversation is moving from the model to the environment it runs in. That’s the right direction.

Sources

SecurityUberAgents