AI models
RMSNorm (Transformer-base)
Training compute
2.8×10¹⁸ FLOP
Parameters
65M
Published
Oct 16, 2019
RMSNorm (Transformer-base) is an AI model developed by University of Edinburgh and University of Zurich (United Kingdom and Switzerland), first published in October 2019. It works in the language domain, on tasks such as translation.
Training it took an estimated 2.8×10¹⁸ FLOP of compute (estimation method: operation counting,hardware). The model has 65,000,000 parameters. It was trained on roughly 7.5B datapoints. Training ran on 1 NVIDIA V100 for about 19.25 hours.
Access: Unreleased. Its weights are not openly released. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- University of Edinburgh, University of Zurich
- Country of organization
- United Kingdom, Switzerland
- Domain
- Language
- Task
- Translation
- Training compute
- 2.8×10¹⁸ FLOP
- Compute estimation method
- Operation counting, Hardware
- Parameters
- 65,000,000
- Dataset size
- 7.5B
- Training hardware
- NVIDIA V100
- Chips used
- 1
- Training time
- 19 h
- Training power draw
- 338 W
- Model accessibility
- Unreleased
- Open weights
- No
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models