Llama 4 Behemoth (preview)

Training compute

5.2×10²⁵ FLOP

Parameters

2T

Published

Apr 5, 2025

Llama 4 Behemoth (preview) is an AI model developed by Meta AI (United States), first published in April 2025. It works in the multimodal, language and vision domain, on tasks such as chat, code generation, visual question answering and 4 more. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 5.2×10²⁵ FLOP of compute (estimation method: operation counting). The model has 2,000,000,000,000 parameters. It was trained on roughly 30T datapoints. Training ran on 32,000 NVIDIA H100 SXM5 80GB. The compute alone is estimated at $45M in 2023 dollars.

Access: Unreleased. Its weights are not openly released. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Meta AI
Country of organization: United States
Domain: Multimodal, Language, Vision
Task: Chat, Code generation, Visual question answering, Translation, Language modeling/generation, Quantitative reasoning, Question answering
Training compute: 5.2×10²⁵ FLOP
Compute estimation method: Operation counting
Parameters: 2,000,000,000,000
Dataset size: 30T
Training hardware: NVIDIA H100 SXM5 80GB
Chips used: 32,000
Training power draw: 43.9 MW
Training cost (2023 USD): $45M
Model accessibility: Unreleased
Open weights: No
Epoch confidence: Likely