AI models
Adaptive Input Transformer + RD
Training compute
8.6×10¹⁹ FLOP
Parameters
247M
Published
Jun 28, 2021
Adaptive Input Transformer + RD is an AI model developed by Microsoft Research Asia and Soochow University (China and Taiwan), first published in June 2021. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 8.6×10¹⁹ FLOP of compute (estimation method: hardware). The model has 247,000,000 parameters. It was trained on roughly 103M datapoints. Training ran on 8 NVIDIA V100 for about 79.4 hours.
Access: Unreleased. Its weights are not openly released. The reference paper has 538 citations. Epoch AI rates the confidence of this record as likely.
Full record
- Organization
- Microsoft Research Asia, Soochow University
- Country of organization
- China, Taiwan
- Domain
- Language
- Task
- Language modeling
- Training compute
- 8.6×10¹⁹ FLOP
- Compute estimation method
- Hardware
- Parameters
- 247,000,000
- Dataset size
- 103M
- Training hardware
- NVIDIA V100
- Chips used
- 8
- Training time
- 79 h
- Training power draw
- 4.9 kW
- Numerical format
- FP16
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 538
- Epoch confidence
- Likely
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models