AI models
T2R + Random Init
Training compute
2.7×10¹⁹ FLOP
Parameters
450M
Published
Mar 24, 2021
T2R + Random Init is an AI model developed by University of Washington, Microsoft, DeepMind and Allen Institute for AI (United States and United Kingdom), first published in March 2021. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 2.7×10¹⁹ FLOP of compute (estimation method: operation counting,hardware). The model has 450,000,000 parameters. It was trained on roughly 103M datapoints. Training ran on 8 NVIDIA V100.
Access: Unreleased. Its weights are not openly released. The reference paper has 94 citations. Epoch AI rates the confidence of this record as likely.
Full record
- Organization
- University of Washington, Microsoft, DeepMind, Allen Institute for AI
- Country of organization
- United States, United Kingdom
- Domain
- Language
- Task
- Language modeling
- Training compute
- 2.7×10¹⁹ FLOP
- Compute estimation method
- Operation counting, Hardware
- Parameters
- 450,000,000
- Dataset size
- 103M
- Training hardware
- NVIDIA V100
- Chips used
- 8
- Chip-hours
- 98
- Training power draw
- 4.9 kW
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 94
- Epoch confidence
- Likely
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models