AI models
Megatron-LM (1T)
Parameters
1T
Published
Apr 9, 2021
Megatron-LM (1T) is an AI model developed by Microsoft Research, NVIDIA and Stanford University (United States), first published in April 2021. It works in the language domain, on tasks such as text autocompletion.
Epoch AI has no training-compute estimate for this model. The model has 1,000,000,000,000 parameters. Training ran on NVIDIA A100.
Access: Unreleased. Its weights are not openly released. The reference paper has 1,172 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Microsoft Research, NVIDIA, Stanford University
- Country of organization
- United States
- Domain
- Language
- Task
- Text autocompletion
- Parameters
- 1,000,000,000,000
- Training hardware
- NVIDIA A100
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 1,172
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models