AI models
T0-XXL
Training compute
9.2×10²⁰ FLOP
Parameters
11B
Published
Oct 15, 2021
T0-XXL is an AI model developed by Hugging Face and Brown University (United States), first published in October 2021. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 9.2×10²⁰ FLOP of compute (estimation method: hardware). The model has 11,000,000,000 parameters. Training ran on 256 Google TPU v3 for about 27 hours. The compute alone is estimated at $12K in 2023 dollars.
Access: Open weights (unrestricted). Its weights are openly available. It is built on top of T5-11B. The reference paper has 1,988 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Hugging Face, Brown University
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling
- Training compute
- 9.2×10²⁰ FLOP
- Compute estimation method
- Hardware
- Parameters
- 11,000,000,000
- Training hardware
- Google TPU v3
- Chips used
- 256
- Training time
- 27 h
- Chip-hours
- 69.1K
- Training power draw
- 232.4 kW
- Training cost (2023 USD)
- $12K
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Base model
- T5-11B
- Citations
- 1,988
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models