Meta Llama 3.1 Debuts the 405B Frontier Open Model

The first open frontier model

Meta released Llama 3.1 on July 23, 2024, publicly releasing Llama 3.1 405B, which it described as the world's largest and most capable openly available foundation model and the first frontier-level open model. Meta said its experimental evaluations showed the flagship competitive with leading models including GPT-4, GPT-4o and Claude 3.5 Sonnet across a range of tasks, evaluated over more than 150 benchmark datasets.

128K context and eight languages

Alongside the 405B, Meta shipped upgraded 8B and 70B models. All were multilingual with support across eight languages and a significantly longer context length of 128K tokens, up from Llama 3's shorter window, enabling use cases like long-form summarization, multilingual agents and coding assistants. Meta also changed its license to allow using Llama outputs, including the 405B's, to improve other models.

Training at scale

Training the 405B on over 15 trillion tokens pushed Meta to more than 16,000 H100 GPUs, the first Llama model trained at that scale. Meta chose a standard decoder-only transformer rather than a mixture-of-experts design for training stability, used an iterative post-training procedure of supervised fine-tuning and direct preference optimization, and quantized the model from 16-bit BF16 to 8-bit FP8 so it could run within a single server node.

The Llama system

Meta released a full reference system with sample applications and new safety components, including the multilingual Llama Guard 3 safety model and Prompt Guard, a prompt injection filter. It also published a request for comment on 'Llama Stack,' a set of standardized interfaces for toolchain components and agentic applications, with over 25 partners including AWS, NVIDIA, Databricks and Google Cloud offering services on day one.

Related on Skillo

See also: Llama 3 and the first open 8B/70B, Llama 3.2 and vision plus edge models.

Sources

• Introducing Llama 3.1 (ai.meta.com)

Published date reflects the original event date (2024-07-23). This article is original Skillo editorial written from the sources above; facts were verified in September 2026.