Skip to content
Docs

Chat SDK with AI Gateway

Turn a Chat SDK bot into an AI assistant with AI Gateway and AI SDK. The handler reads recent conversation history, requests a model response, and streams the answer back to the thread.

Prerequisites

The examples use the quickstart project layout with src/lib/bot.ts. For a project without src, place the files under lib instead.

Configure AI Gateway

Install AI SDK:

Terminal
pnpm add ai

Choose a model ID and set AI_GATEWAY_MODEL in your local and deployed environment variables. For example:

.env.local
AI_GATEWAY_MODEL=anthropic/claude-opus-5.5

On Edgenivo, AI SDK can authenticate through the deployment OIDC token. For local development, link the project and run vercel env pull .env.local to obtain a development token. You can also set AI_GATEWAY_API_KEY. See AI Gateway authentication for setup and precedence.

Stream a response with conversation history

Create src/lib/respond.ts. Read history through thread.messages so adapters without a platform history API can use Chat SDK's stored messages. toAiMessages maps authors to model roles and sorts the messages oldest first:

src/lib/respond.ts
import { streamText } from 'ai';
import type { Message, Thread } from 'chat';
import { toAiMessages } from 'chat/ai';
 
export async function respond(thread: Thread, _message: Message) {
  const model = process.env.AI_GATEWAY_MODEL;
  if (!model)
    throw new Error('Set AI_GATEWAY_MODEL to an AI Gateway model ID.');
 
  const history: Message[] = [];
  for await (const message of thread.messages) {
    history.push(message);
    if (history.length === 20) break;
  }
 
  const result = streamText({
    model,
    system: 'Answer questions using the conversation for context.',
    messages: await toAiMessages(history),
    abortSignal: AbortSignal.timeout(60_000),
  });
 
  await thread.post(result.textStream);
}

Await thread.post so the handler stays active until the stream finishes. The framework webhook route must still pass its background processing callback, as shown in the framework guides.

Register the response handler

In src/lib/bot.ts, keep your existing bot configuration and replace its echo handlers with the following registrations:

src/lib/bot.ts
import { respond } from './respond';
 
// Keep your existing Chat instance, adapters, and state configuration.
 
bot.onNewMention(async (thread, message) => {
  await thread.subscribe();
  await respond(thread, message);
});
bot.onSubscribedMessage(respond);
bot.onDirectMessage(respond);

Test the bot

Deploy the updated bot, invite it to a channel, and mention it with a question. Ask a follow-up in the same thread. The bot should use the earlier messages when answering.

Platforms differ in how they display streamed replies. Use the Web Adapter with protocol: 'ai-sdk' to serve a browser chat from the same handler.

The handler reads at most 20 messages; it does not summarize older conversations. The model request has a 60-second timeout, and the webhook function's maximum duration must also cover history retrieval and delivery.

Next steps

Previous
Integrations
Next
Edgenivo Connect

Was this helpful?

supported.