AI models
FMMformer (2-kernel fast weight + Band20)
Training compute
2.5×10¹⁶ FLOP
Parameters
40M
Published
Aug 5, 2021
FMMformer (2-kernel fast weight + Band20) is an AI model developed by University of California Los Angeles (UCLA) and University of Utah (United States), first published in August 2021. It works in the language domain, on tasks such as language modeling and text classification.
Training it took an estimated 2.5×10¹⁶ FLOP of compute (estimation method: operation counting). The model has 40,000,000 parameters. Training ran on 4 NVIDIA GeForce RTX 3090 Ti.
Access: Unreleased. Its weights are not openly released. The reference paper has 38 citations. Epoch AI rates the confidence of this record as speculative.
Full record
- Organization
- University of California Los Angeles (UCLA), University of Utah
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling, Text classification
- Training compute
- 2.5×10¹⁶ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 40,000,000
- Training hardware
- NVIDIA GeForce RTX 3090 Ti
- Chips used
- 4
- Training power draw
- 3.6 kW
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 38
- Epoch confidence
- Speculative
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models