AI models
Memformer (4 encoder + 16 decoder)
Training compute
1.2×10¹⁹ FLOP
Parameters
76.2M
Published
Oct 14, 2020
Memformer (4 encoder + 16 decoder) is an AI model developed by UC Davis, Westlake University and Facebook AI (United States and China), first published in October 2020. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 1.2×10¹⁹ FLOP of compute (estimation method: hardware). The model has 76,200,000 parameters. It was trained on roughly 103M datapoints. Training ran on 4 NVIDIA Tesla V100 DGXS 16 GB,NVIDIA GeForce RTX 2080 Ti 11GB for about 96 hours.
Access: Unreleased. Its weights are not openly released. The reference paper has 77 citations. Epoch AI rates the confidence of this record as likely.
Full record
- Organization
- UC Davis, Westlake University, Facebook AI
- Country of organization
- United States, China
- Domain
- Language
- Task
- Language modeling
- Training compute
- 1.2×10¹⁹ FLOP
- Compute estimation method
- Hardware
- Parameters
- 76,200,000
- Dataset size
- 103M
- Training hardware
- NVIDIA Tesla V100 DGXS 16 GB, NVIDIA GeForce RTX 2080 Ti 11GB
- Chips used
- 4
- Training time
- 96 h
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 77
- Epoch confidence
- Likely
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models