AI models
T2R 75% + Pretrain (WT-103)
Training compute
1.4×10¹⁹ FLOP
Parameters
668.9M
Published
Mar 24, 2021
T2R 75% + Pretrain (WT-103) is an AI model developed by University of Washington, Microsoft and DeepMind (United States and United Kingdom), first published in March 2021. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 1.4×10¹⁹ FLOP of compute (estimation method: operation counting,hardware). The model has 668,893,184 parameters. Training ran on 8 NVIDIA V100 for about 11.8 hours.
Access: Unreleased. Its weights are not openly released. The reference paper has 94 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- University of Washington, Microsoft, DeepMind
- Country of organization
- United States, United Kingdom
- Domain
- Language
- Task
- Language modeling
- Training compute
- 1.4×10¹⁹ FLOP
- Compute estimation method
- Operation counting, Hardware
- Parameters
- 668,893,184
- Training hardware
- NVIDIA V100
- Chips used
- 8
- Training time
- 12 h
- Training power draw
- 4.9 kW
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 94
- Epoch confidence
- Confident
More from University of Washington,Microsoft,DeepMind
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models