AI models
OLMoE
Training compute
5.2×10²² FLOP
Parameters
7B
Published
Sep 3, 2024
OLMoE is an AI model developed by Allen Institute for AI, Contextual AI, University of Washington and Princeton University (United States), first published in September 2024. It works in the language domain, on tasks such as language modeling/generation and chat.
Training it took an estimated 5.2×10²² FLOP of compute (estimation method: operation counting,hardware). The model has 7,000,000,000 parameters. Training ran on 256 NVIDIA H100 SXM5 80GB for about 287 hours.
Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Allen Institute for AI, Contextual AI, University of Washington, Princeton University
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling/generation, Chat
- Training compute
- 5.2×10²² FLOP
- Compute estimation method
- Operation counting, Hardware
- Parameters
- 7,000,000,000
- Training hardware
- NVIDIA H100 SXM5 80GB
- Chips used
- 256
- Training time
- 287 h
- Training power draw
- 353.1 kW
- Numerical format
- BF16
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models