AI models
Fraternal dropout + AWD-LSTM 3-layer (WT2)
Training compute
3.1×10¹⁷ FLOP
Parameters
34M
Published
Oct 31, 2017
Fraternal dropout + AWD-LSTM 3-layer (WT2) is an AI model developed by Jagiellonian University, Mila - Quebec AI (originally Montreal Institute for Learning Algorithms) and University of Montreal / Université de Montréal (Poland and Canada), first published in October 2017. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 3.1×10¹⁷ FLOP of compute (estimation method: operation counting). The model has 34,000,000 parameters. It was trained on roughly 2M datapoints.
Access: Unreleased. Its weights are not openly released. The reference paper has 55 citations. Epoch AI rates the confidence of this record as likely.
Full record
- Organization
- Jagiellonian University, Mila - Quebec AI (originally Montreal Institute for Learning Algorithms), University of Montreal / Université de Montréal
- Country of organization
- Poland, Canada
- Domain
- Language
- Task
- Language modeling
- Training compute
- 3.1×10¹⁷ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 34,000,000
- Dataset size
- 2M
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 55
- Epoch confidence
- Likely
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models