How to control AI provider data handling with AI Gateway

Use Edgenivo AI Gateway to restrict which providers and models receive your requests and enforce provider training and retention requirements. You control what enters each prompt and what your application logs. Review provider-specific exceptions before sending sensitive data.

Copy link to headingStart with the data you need to protect

Before choosing a model, identify what your AI feature needs to process. An internal support assistant may need the customer's problem description, product version, and relevant troubleshooting instructions.

The full ticket can also contain names, email addresses, account identifiers, attachments, or credentials pasted into error reports.

Send the information needed to answer the question. For a connection timeout, include the error and product version. Leave out unrelated details, such as the customer’s billing address.

Review internal documents before adding them to a prompt. Remove proprietary procedures, infrastructure details, or secrets that the assistant doesn't need.

Define a policy for the feature before implementation:

• Record permitted data categories, approved providers and models, and training and retention requirements.
• Define what the application should do when no approved route is available.
• Assign an owner to review the policy when the app or provider catalog changes.

Copy link to headingTrace where request data goes

Requests typically follow this path:

User > application > AI Gateway > inference provider > application

Your application constructs the prompt. AI Gateway selects an eligible provider, forwards the inference request, and returns the response.

Retrieval can add documents from a database, and tools can fetch records or call external services. Application logs, stored conversations, and telemetry can retain additional copies of request data.

Check three requirements at each destination:

Copy link to headingNo training doesn’t mean no retention

Providers can prohibit training while retaining content for a feature or safety review. Anthropic's retention policy defines which API features and models qualify for zero data retention (ZDR).

Review the model, serving provider, API feature, and credentials together. The organization hosting inference may differ from the model’s creator. Consumer chat settings do not determine the terms for your application’s API requests.

Copy link to headingPrevent training and enforce zero data retention

Under its security and compliance policy, AI Gateway permanently deletes prompts and responses after requests complete. Configure provider-level routing controls to enforce training and retention requirements for upstream providers.

Configure AI Gateway’s training and ZDR controls for your workload:

• Prevent prompt training: disallowPromptTraining: true limits requests using AI Gateway system credentials to providers with no-training agreements. Your own provider keys follow your provider agreement and settings.
• Require zero data retention: Set zeroDataRetention: true to require providers covered by ZDR agreements.

ZDR includes no-training controls, so you don't need to enable both.

Require ZDR for every support assistant request. To enable it team-wide, open AI Gateway > Settings in the Edgenivo dashboard and enable Zero Data Retention. You can also enforce ZDR on individual requests in code.

ZDR applies when either the team setting or the request enables it, so individual requests cannot override team-wide enforcement. AI Gateway returns an error if no eligible route remains after considering fallback providers.

Review the selected provider’s ZDR policy for model and feature exceptions, including prompt caching. AI Gateway can accept requests that use features excluded from that policy. Prevent your application from using those features when your workload requires no retention.

Before using this server-side example, install AI SDK 7 and configure AI_GATEWAY_API_KEY using the AI Gateway text-generation quickstart.

Require ZDR and omit prompts and responses from SDK telemetry:

import { generateText } from 'ai';

const model = process.env.SUPPORT_AI_MODEL;

if (!model) {
  throw new Error('Configure an approved support model.');
}

const result = await generateText({
  model,
  instructions: 'Summarize the issue and suggest a troubleshooting step.',
  prompt: 'The desktop client times out after 30 seconds.',
  providerOptions: {
    gateway: {
      zeroDataRetention: true,
    },
  },
  telemetry: {
    recordInputs: false,
    recordOutputs: false,
  },
});

Set SUPPORT_AI_MODEL to an approved AI Gateway model ID with an eligible route.

Handle unavailable approved providers as an expected error. Offer manual support or retry another approved route with the same restrictions. Keep those restrictions in place in every error handler.

Copy link to headingReview agreements for your own provider keys

Bring your own key (BYOK) requests operate under your provider agreement. Under AI Gateway’s BYOK and ZDR rules, AI Gateway skips those keys when ZDR is required unless you mark them as ZDR-compliant.

Confirm that your provider agreement covers ZDR before marking a key as compliant. Marking the key makes it eligible for ZDR routing; you still need to enable ZDR for the team or request. Record who reviewed the agreement and which workloads it covers.

Copy link to headingLimit which providers and models can serve requests

Security teams may review contractual terms, hosting arrangements, incident processes, and suitability for the data alongside retention.

AI Gateway provides separate controls for providers and models:

Enable the allowlists in AI Gateway > Settings and disable unapproved providers and models. All existing entries start enabled to preserve current traffic when you turn on an allowlist.

Newly added providers and models start disabled. Changes can take a couple of minutes to propagate; wait before testing.

Approve the serving provider and model separately. Blocking one provider does not necessarily block a model that another allowed provider can serve.

To narrow routing for a workload, add an only list to the AI Gateway provider options. Providers must appear in both that list and the team’s provider allowlist. An order list sets preferences but can leave other providers available as fallbacks. Use only to restrict destinations.

Keep the support assistant's model choice in server-side configuration. Map any user-selectable options to approved model IDs and fallback configurations.

These controls apply to requests routed through AI Gateway. Review direct provider integrations, background jobs, and external tools separately, and apply the same data-handling requirements to those paths.

Copy link to headingReview data location requirements

ZDR does not determine data location. Where supported, AI Gateway regional inference lets you choose the provider's inference region and configures any storage of request data in the same region. Requests fail when the selected region cannot be honored.

Provider policies can make exceptions for abuse or safety reviews, including retaining flagged requests outside the selected region. Check those terms before relying on regional routing for residency requirements.

This setting does not restrict AI Gateway processing to the selected region. AI Gateway may receive and process the request in another Edgenivo region before forwarding it to the provider.

Evaluate the complete path when a workload has residency requirements, including application storage, retrieval, tools, and telemetry exports.

Copy link to headingTest your implementation

Verify which routes served your requests and which restrictions applied. Use routing metadata to document enforcement without retaining conversation content.

Use AI Gateway request logs to inspect the serving model and provider, authentication source, ZDR status, and routing attempts. Open a request to check for failover and see which credentials served each attempt. Export the evidence you need before the routing attempt details expire after 30 days.

On Pro and Enterprise, AI Gateway Trace Drains send request metadata to an OpenTelemetry Protocol over HTTP (OTLP/HTTP) endpoint. Traces omit prompt and completion content. Delivery and data transfer incur usage charges.

AI SDK 7 telemetry events include inputs and outputs by default. Data reaches your observability tools through registered or per-call integrations; Node.js tracing-channel subscribers can receive events too.

The earlier example excludes content with recordInputs: false and recordOutputs: false. To turn off all telemetry events for a call, including tracing-channel events, set telemetry: { isEnabled: false }.

const result = await generateText({
  // ...
  telemetry: {
    isEnabled: false,
  },
});

Review custom logs, error handlers, and reporting metadata for ticket content recorded outside the SDK controls. Use request IDs and non-sensitive operational labels to investigate failures.

Before release, test with synthetic data:

1. Confirm an approved model succeeds through an approved, eligible provider.
2. Confirm that an unapproved model is blocked.
3. Confirm that a request restricted to an unapproved provider is blocked.
4. Confirm that a request with no eligible route fails without relaxing policy.
5. Check that application logs and telemetry omit ticket content.

Review routing evidence for successful requests and recovery paths. Receiving a successful response does not establish which controls were applied.

Copy link to headingNext steps

After release, revisit approved providers and models whenever the application, provider catalog, or agreements change. Check routing logs to confirm that requests still comply with your policy.

When data-handling requirements change, update retention settings for stored conversations and telemetry along with customer disclosures.