AI models
DEQ-Transformer (Medium, Adaptive Embedding)
Training compute
8.2×10¹⁷ FLOP
Parameters
110M
Published
Sep 3, 2019
DEQ-Transformer (Medium, Adaptive Embedding) is an AI model developed by Carnegie Mellon University (CMU) and Intel Labs (United States), first published in September 2019. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 8.2×10¹⁷ FLOP of compute (estimation method: operation counting). The model has 110,000,000 parameters. It was trained on roughly 103M datapoints.
Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 873 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Carnegie Mellon University (CMU), Intel Labs
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling
- Training compute
- 8.2×10¹⁷ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 110,000,000
- Dataset size
- 103M
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Citations
- 873
- Epoch confidence
- Confident
More from Carnegie Mellon University (CMU),Intel Labs
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models