Streaming

OpenResponses Streaming with AI Gateway

The OpenResponses API supports streaming to receive tokens as they're generated instead of waiting for the complete response. Set stream: true in your request, then read the response body as a stream of server-sent events. Each event contains a response chunk that you can display incrementally.

stream.ts

const apiKey = process.env.AI_GATEWAY_API_KEY;
 
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify({
    model: 'google/gemini-3.1-pro-preview',
    input: [
      {
        type: 'message',
        role: 'user',
        content: 'Write a haiku about debugging code.',
      },
    ],
    stream: true,
  }),
});
 
if (!response.ok || !response.body) throw new Error(await response.text());
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
 
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
 
  buffer += decoder.decode(value, { stream: true });
 
  // Keep the trailing fragment in the buffer: a read can end mid-line, and
  // parsing a half-received event throws.
  const lines = buffer.split('\n');
  buffer = lines.pop() ?? '';
 
  for (const line of lines) {
    if (!line.startsWith('data:')) continue;
    const data = line.slice(5).trim();
    if (!data || data === '[DONE]') continue;
 
    const event = JSON.parse(data);
    if (event.type === 'response.output_text.delta') {
      process.stdout.write(event.delta);
    }
  }
}

stream.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
stream = client.responses.create(
    model='google/gemini-3.1-pro-preview',
    input=[
        {
            'type': 'message',
            'role': 'user',
            'content': 'Write a haiku about debugging code.',
        },
    ],
    stream=True,
)
 
for event in stream:
    if event.type == 'response.output_text.delta':
        print(event.delta, end='', flush=True)

stream.sh

curl -X POST "https://ai-gateway.vercel.sh/v1/responses" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.1-pro-preview",
    "input": [
      {
        "type": "message",
        "role": "user",
        "content": "Write a haiku about debugging code."
      }
    ],
    "stream": true
  }'

Streaming events

response.created - Response initialized
response.output_text.delta - Text chunk received
response.output_text.done - Text generation complete
response.completed - Full response complete with usage stats

Last updated September 8, 2026

Text Generation

Tool Calling

Was this helpful?