# Llama 3.1 70B Instruct

Llama 3.1 70B Instruct was the first open 70B model to combine a context window of 128K tokens, multilingual fluency across eight languages, and trained tool-use capability in a single package. Every subsequent Llama 70B iteration builds on this baseline.

- **Model ID:** `meta/llama-3.1-70b`
- **Type:** chat
- **Providers:** bedrock
- **Context window:** 128,000
- **Maximum output tokens:** 8,192
- **Pricing:** $0.72/1M input tokens, $0.72/1M output tokens
- **Canonical page:** https://vercel.com/ai-gateway/models/llama-3.1-70b

## Supported parameters

Detailed capability metadata has not been reported for this model.

## Example

```ts
import { streamText } from 'ai'

const result = streamText({
  model: 'meta/llama-3.1-70b',
  prompt: 'Why is the sky blue?'
})
```

## About

Meta released Llama 3.1 70B Instruct on July 23, 2024. The 70B variant sat in the middle of a three-model release (8B, 70B, 405B) but carried the full 3.1 feature set: a context window of 128K tokens (16 times longer than Llama 3's limit), multilingual support across eight languages, and tool-use capability trained directly into the model rather than bolted on through prompting tricks.

The eight-language support (English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai) goes beyond surface-level translation. Llama 3.1 70B Instruct matched or beat peer open and closed models in its parameter class on more than 150 benchmark datasets spanning these languages. A single model can handle cross-language conversations without maintaining separate language-specific deployments.

Tool use was a first for the Llama family at this scale. The 3.1 generation was explicitly trained for agentic workflows: calling APIs, querying databases, and invoking custom functions as part of a reasoning chain. Meta's Llama Stack API, introduced alongside 3.1, standardized the interfaces for connecting the model to external toolchains and retrieval-augmented generation (RAG) systems.

Meta also expanded the open-weight license in 3.1. You can use model outputs to train or improve other models.

## What to consider

For workloads that push the full context window of 128K tokens (processing entire codebases or book-length documents), provider differences in long-context throughput become significant. Benchmark with representative payloads before you commit. Compare $0.72 and $0.72.

## When to use

### Best For

- Long-context workloads that need the full window of 128K tokens, processing legal documents, technical manuals, or entire codebases in a single pass without chunking
- Multilingual applications where a single model must handle conversations and instructions across English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai
- Agentic systems that require trained tool-use capability for calling external APIs, databases, and custom functions as part of multi-step reasoning

### Consider Alternatives When

- A smaller, faster model satisfies quality needs; Llama 3.1 8B is the lighter option within the same generation
- Vision or image understanding is required; Llama 3.2 11B or 90B provide native multimodal input
- Maximum instruction-following quality at 70B is the priority and serving cost allows for a newer 70B release such as Llama 3.3 70B

## Best for

- **Long-context workloads:** Processing legal documents, technical manuals, or entire codebases within the full window of 128K tokens in a single pass without chunking
- **Multilingual applications:** A single model handles conversations and instructions across English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai
- **Agentic systems:** Trained tool-use capability for calling external APIs, databases, and custom functions as part of multi-step reasoning

## Consider alternatives

- **Smaller model sufficient:** A smaller, faster model satisfies quality needs, so Llama 3.1 8B is the lighter option within the same generation
- **Vision required:** Image understanding is required, so Llama 3.2 11B or 90B provide native multimodal input
- **Refined 70B preferred:** Maximum instruction-following quality at 70B is the priority and serving cost allows for a newer 70B release such as Llama 3.3 70B

## Frequently asked questions

### Why was the jump to context of 128K tokens significant in Llama 3.1 70B Instruct?

Llama 3 supported only 8K context. The 16x expansion to 128K tokens in Llama 3.1 70B Instruct lets you process entire codebases, book-length documents, and multi-hour transcripts in a single pass. This eliminates the chunking and retrieval complexity that shorter context windows require.

### Which eight languages does Llama 3.1 70B Instruct support?

English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. Llama 3.1 70B Instruct was evaluated across more than 150 benchmark datasets spanning these languages.

### What does trained tool use mean in practice?

The model calls external functions, APIs, and databases as part of its reasoning process. Unlike prompt-engineered tool use, this capability was learned during training, making function calls more reliable and less dependent on careful prompt construction.

### What is Llama Stack and how does it relate to the 70B?

Llama Stack is Meta's standardized set of interfaces for retrieval-augmented generation and agentic application development. It was introduced alongside the 3.1 generation and provides a consistent integration layer for connecting the model to external tools and data sources.

## Links

- [Model page](https://vercel.com/ai-gateway/models/llama-3.1-70b)
- [AI Gateway documentation](https://vercel.com/docs/ai-gateway)
- [Provider model documentation](https://aws.amazon.com/bedrock/meta/)
- [Provider pricing](https://aws.amazon.com/bedrock/pricing/)


If you encounter incorrect information, can’t find what you need, need a capability Vercel doesn’t currently offer, or have any other feedback to report, please do it by following the instructions at https://vercel.com/feedback.md.