AI models
DEQ-Transformer (Post-LN) + Jacobian Regularisation
Training compute
2.9×10¹⁹ FLOP
Parameters
98M
Published
Jun 28, 2021
DEQ-Transformer (Post-LN) + Jacobian Regularisation is an AI model developed by Carnegie Mellon University (CMU) and Intel Labs (United States), first published in June 2021. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 2.9×10¹⁹ FLOP of compute (estimation method: operation counting,hardware). The model has 98,000,000 parameters. Training ran on 4 NVIDIA GeForce RTX 2080 Ti 11GB for about 250 hours.
Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 81 citations. Epoch AI rates the confidence of this record as likely.
Full record
- Organization
- Carnegie Mellon University (CMU), Intel Labs
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling
- Training compute
- 2.9×10¹⁹ FLOP
- Compute estimation method
- Operation counting, Hardware
- Parameters
- 98,000,000
- Training hardware
- NVIDIA GeForce RTX 2080 Ti 11GB
- Chips used
- 4
- Training time
- 250 h
- Training power draw
- 2.0 kW
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Citations
- 81
- Epoch confidence
- Likely
More from Carnegie Mellon University (CMU),Intel Labs
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models