AI models
Transformer-XL Large + Phrase Induction
Training compute
3.8×10²⁰ FLOP
Parameters
257M
Published
Jun 4, 2019
Transformer-XL Large + Phrase Induction is an AI model developed by Massachusetts Institute of Technology (MIT) and University of Illinois Urbana-Champaign (UIUC) (United States), first published in June 2019. It works in the language domain, on tasks such as language modeling/generation.
Training it took an estimated 3.8×10²⁰ FLOP of compute (estimation method: operation counting). The model has 257,000,000 parameters. It was trained on roughly 103M datapoints.
Access: Unreleased. Its weights are not openly released. It is built on top of Transformer-XL (257M). The reference paper has 14 citations. Epoch AI rates the confidence of this record as speculative.
Full record
- Organization
- Massachusetts Institute of Technology (MIT), University of Illinois Urbana-Champaign (UIUC)
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling/generation
- Training compute
- 3.8×10²⁰ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 257,000,000
- Dataset size
- 103M
- Model accessibility
- Unreleased
- Open weights
- No
- Base model
- Transformer-XL (257M)
- Citations
- 14
- Epoch confidence
- Speculative
More from Massachusetts Institute of Technology (MIT),University of Illinois Urbana-Champaign (UIUC)
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models