OpenAI Chat Completions Requests with AI Gateway

Create chat completions using various AI models available through the AI Gateway.

Create a non-streaming chat completion.

import OpenAI from 'openai';
 
const apiKey = process.env.AI_GATEWAY_API_KEY || process.env.VERCEL_OIDC_TOKEN;
 
const openai = new OpenAI({
  apiKey,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const completion = await openai.chat.completions.create({
  model: 'anthropic/claude-opus-5',
  messages: [
    {
      role: 'user',
      content: 'Write a one-sentence bedtime story about a unicorn.',
    },
  ],
  stream: false,
});
 
console.log('Assistant:', completion.choices[0].message.content);
console.log('Tokens used:', completion.usage);

import os
from openai import OpenAI
 
api_key = os.getenv('AI_GATEWAY_API_KEY') or os.getenv('VERCEL_OIDC_TOKEN')
 
client = OpenAI(
    api_key=api_key,
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
completion = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {
            'role': 'user',
            'content': 'Write a one-sentence bedtime story about a unicorn.'
        }
    ],
    stream=False,
)
 
print('Assistant:', completion.choices[0].message.content)
print('Tokens used:', completion.usage)

curl -X POST "https://ai-gateway.vercel.sh/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-opus-5",
    "messages": [
      {
        "role": "user",
        "content": "Write a one-sentence bedtime story about a unicorn."
      }
    ],
    "stream": false
  }'

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "anthropic/claude-opus-5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Once upon a time, a gentle unicorn with a shimmering silver mane danced through moonlit clouds, sprinkling stardust dreams upon sleeping children below."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 28,
    "total_tokens": 43
  }
}

Set stream: true to receive tokens as they are generated. See Streaming for the full example and the server-sent event format.

Send images and PDF documents by using an array of content parts in place of a plain string. See File attachments for the full examples and supported types.

The chat completions endpoint supports the following parameters:

• model (string): The model to use for the completion (e.g., anthropic/claude-opus-5)
• messages (array): Array of message objects with role and content fields

• stream (boolean): Whether to stream the response. Defaults to false
• temperature (number): Controls randomness in the output. Range: 0-2
• max_tokens (integer): Maximum number of tokens to generate
• top_p (number): Nucleus sampling parameter. Range: 0-1
• frequency_penalty (number): Penalty for frequent tokens. Range: -2 to 2
• presence_penalty (number): Penalty for present tokens. Range: -2 to 2
• stop (string or array): Stop sequences for the generation
• safety_identifier (string): A non-empty, opaque ID for the end user. AI Gateway truncates values longer than 64 characters and maps the field to providerOptions.gateway.safetyIdentifier. See Safety identifiers for examples and precedence
• tools (array): Array of tool definitions for function calling
• tool_choice (string or object): Controls which tools are called (auto, none, or specific function)
• providerOptions (object): Provider routing and configuration options
• provider (object): Shorthand for provider routing. Supports sort ('cost', 'ttft', 'tps'). Equivalent to setting providerOptions.gateway.sort. See Provider sorting
• response_format (object): Controls the format of the model's response
  • For OpenAI standard format: { type: "json_schema", json_schema: { name, schema, strict?, description? } }
  • For legacy format: { type: "json", schema?, name?, description? }
  • For plain text: { type: "text" }
  • See Structured outputs for detailed examples

Messages support different content types:

{
  "role": "user",
  "content": "Hello, how are you?"
}

{
  "role": "user",
  "content": [
    { "type": "text", "text": "What's in this image?" },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD..."
      }
    }
  ]
}

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Summarize this document" },
    {
      "type": "file",
      "file": {
        "data": "JVBERi0xLjQKJcfsj6IKNSAwIG9iago8PAovVHlwZSAvUGFnZQo...",
        "media_type": "application/pdf",
        "filename": "document.pdf"
      }
    }
  ]
}