Flue with AI Gateway

Flue agents can use AI Gateway through the built-in vercel-ai-gateway provider. Choose a model with useModel() to route requests through AI Gateway and track usage in your Edgenivo dashboard.

Start with an existing Flue project with @flue/runtime and @flue/cli installed, Node.js 22.19 or later, and a Edgenivo account.

Create an AI Gateway API key and add it to .env.local:

AI_GATEWAY_API_KEY=your-api-key-here

For deployed applications, set AI_GATEWAY_API_KEY in your project's environment variables. To authenticate with a Edgenivo OpenID Connect (OIDC) token instead, use the OIDC configuration below.

Create or update your agent module:

'use agent';
 
import { useModel } from '@flue/runtime';
 
export function Assistant() {
  useModel('vercel-ai-gateway/openai/gpt-6.1-sol');
  return 'Answer questions clearly and concisely.';
}

The first part of the model specifier, vercel-ai-gateway, selects Flue's provider. The remaining openai/gpt-6.1-sol is the AI Gateway model ID. To use another model, choose one from the provider's model catalog.

Flue includes all built-in providers by default. If your Flue configuration has a providers list, include 'vercel-ai-gateway' in that list for server builds. See Flue's provider configuration.

Load .env.local and send a prompt with the Flue CLI:

pnpm exec flue run src/agents/assistant.ts --env .env.local --message "Explain AI Gateway in one sentence."

For the JavaScript example, use src/agents/assistant.js. After the response, check your request in AI Gateway observability.

Flue's built-in AI Gateway provider reads AI_GATEWAY_API_KEY. To use a Edgenivo OIDC token, register a provider with a request-time authentication resolver. This sends the token in the Authorization header to AI Gateway's Anthropic-compatible Messages API.

Install the provider and OIDC packages:

pnpm i @earendil-works/pi-ai @vercel/oidc

yarn add @earendil-works/pi-ai @vercel/oidc

npm i @earendil-works/pi-ai @vercel/oidc

bun add @earendil-works/pi-ai @vercel/oidc

For local development, link your project and pull a token:

vercel link
vercel env pull .env.local

Replace the agent module with the following configuration. Keep the registration in the agent module so it also applies to flue run, which doesn't load app.ts:

'use agent';
 
import { createProvider } from '@earendil-works/pi-ai';
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy';
import { vercelAIGatewayProvider } from '@earendil-works/pi-ai/providers/vercel-ai-gateway';
import { setProvider, useModel } from '@flue/runtime';
import { getVercelOidcToken } from '@vercel/oidc';
 
setProvider(
  createProvider({
    id: 'vercel-ai-gateway',
    baseUrl: 'https://ai-gateway.vercel.sh',
    models: vercelAIGatewayProvider().getModels(),
    api: anthropicMessagesApi(),
    auth: {
      apiKey: {
        name: 'Edgenivo OIDC',
        resolve: async () => ({
          auth: {
            headers: {
              Authorization: `Bearer ${await getVercelOidcToken()}`,
            },
          },
        }),
      },
    },
  }),
);
 
export function Assistant() {
  useModel('vercel-ai-gateway/openai/gpt-6.1-sol');
  return 'Answer questions clearly and concisely.';
}

The registration replaces the built-in provider's authentication for chat requests while retaining its model catalog. It uses OIDC even if AI_GATEWAY_API_KEY is set. getVercelOidcToken() resolves the current token when authentication runs, rather than capturing it when the module loads.

On Edgenivo, your deployment supplies the OIDC token. For local development, rerun vercel env pull .env.local when the token expires. See AI Gateway OIDC authentication for details. Run the agent with the same CLI command above.

To give your agent isolated file and command tools, follow Flue with Edgenivo Sandbox. Model requests continue to use AI Gateway from your application process.