What Actually Happens to a Prompt
When a user submits a prompt to an enterprise AI system, the text leaves the client, travels to the provider's API endpoint, gets processed by the model, and a completion comes back. That's the part everyone pictures. What's less visible: the prompt and completion may be logged at the API gateway layer, retained in the provider's abuse-monitoring infrastructure, written to regional processing nodes before the response returns, and stored in session buffers that persist beyond the immediate exchange.
"May be" is doing real work in that sentence. Whether any of those things happen depends on the provider, the tier of service, the enterprise agreement in place, and which regional endpoint the traffic is routed to. The default behavior for a developer API key is not the same as the behavior under an enterprise data processing addendum. The consumer product and the enterprise product are governed by different agreements with materially different data handling terms — and legal teams need to start there.
Zero Data Retention: The Contractual Reality
Enterprise-tier AI agreements from major providers — OpenAI, Anthropic, Google, Microsoft — typically include what's marketed as zero data retention (ZDR). The commitment: the provider will not store prompt or completion data after processing. No training on your data. No retention for abuse review beyond the immediate session. Prompts in, response out, nothing kept.
That commitment is real, and it is entirely contractual.
The provider is making a promise about what their infrastructure does with your data. You cannot independently verify that the promise is being kept. You cannot audit the provider's logging configuration. You cannot demonstrate to an auditor that no copy exists by showing them a control — you can only show them the contract. When the CISO says "I need this in my control inventory," a ZDR commitment answers a different question. The CISO wants a technical state; ZDR documents a vendor obligation. Those are not the same thing, and in a compliance conversation, the gap between them is where the argument starts.
In federal contexts, the gap is acute. An agency operating under FedRAMP Moderate or High authorization needs to demonstrate that controls are implemented, not merely promised. A ZDR clause in an enterprise agreement is evidence of a vendor commitment; it is not a control implementation. Legal teams and CISOs stop agreeing at exactly this point, and your conversation needs to be precise about why.
Regional Endpoints and What "Residency" Actually Means
Data residency requirements — common in EU deployments under GDPR, and in US federal deployments under data sovereignty requirements — are addressed by providers through regional endpoints. Traffic routed to an EU endpoint is processed in EU infrastructure; the data nominally doesn't cross to US processing nodes. US-Gov endpoints route to FedRAMP-authorized infrastructure with additional access controls. Sovereign cloud deployments (Azure Government, Google Distributed Cloud) go further, running on infrastructure that's physically and logically separated from the commercial cloud.
Regional endpoints address where processing happens. They don't automatically address retention policy, logging behavior, or training data use — those are governed by the enterprise agreement, which may or may not vary by region. A provider can route your traffic to an EU endpoint and still retain logs under a default retention policy unless the enterprise agreement explicitly overrides it. Residency and retention are separate questions that happen to live in the same conversation.
Legal teams end up maintaining what amounts to a matrix: providers crossed against regions crossed against retention policies, updated as agreements change. It's not a one-time exercise. Provider terms evolve, regional infrastructure expands, and enterprise agreement renewals are opportunities for terms to shift in either direction.
Okta Concept Mapping
The natural IDAM analogy here is the data processing addendum you negotiate with a third-party IdP — the agreement governing what the IdP logs about your users' authentication events, how long they keep it, and who can access it. You've had this conversation. The break point: in federated identity, the enterprise typically maintains its own copy of authentication events in its SIEM. If the IdP's logs are ever in question, you have an independent record. Under AI ZDR, the enterprise may have no independent copy of the prompt at all. The only record of what was sent is what the provider says they deleted. That asymmetry is new, and it's what makes ZDR harder to operationalize than a standard DPA.
What the Subpoena Question Actually Surfaces
If ZDR is functioning as contracted, a subpoena to the AI provider produces nothing — there's nothing retained to produce. Under the enterprise agreement, that's the intended outcome. The practical question is whether the enterprise has its own logs. If the agency is running prompts through a gateway with logging enabled, those logs exist on agency infrastructure and are fully discoverable. If the agency disabled gateway logging to reduce data exposure, the tradeoff is that there's no audit trail for the agency's own compliance purposes either.
No clean answer exists here. It's a tradeoff that legal, compliance, and security need to make together, with full understanding of what each configuration preserves and what it sacrifices. Your job in that conversation is to make sure the tradeoff is explicit — that nobody is assuming ZDR means "no logs anywhere" when what it means is "no logs at the provider, under contract."
The agencies that handle this well are the ones that mapped the question before the incident. The ones that handle it poorly are the ones who discover the gap when someone asks what the AI system was told last March.