AI models
$\infty$-former (SM)
Training compute
1.2×10²² FLOP
Parameters
124M
Published
Sep 1, 2021
$\infty$-former (SM) is an AI model developed by Universidade de Lisboa (ULisboa) and DeepMind (Portugal and United Kingdom), first published in September 2021. It works in the language domain, on tasks such as language modeling/generation.
Training it took an estimated 1.2×10²² FLOP of compute (estimation method: operation counting). The model has 124,000,000 parameters. It was trained on roughly 200M datapoints. Training ran on 1 NVIDIA GeForce RTX 2080 Ti 11GB.
Access: Unreleased. Its weights are not openly released. It is built on top of GPT-2 (124M). The reference paper has 31 citations. Epoch AI rates the confidence of this record as speculative.
Full record
- Organization
- Universidade de Lisboa (ULisboa), DeepMind
- Country of organization
- Portugal, United Kingdom
- Domain
- Language
- Task
- Language modeling/generation
- Training compute
- 1.2×10²² FLOP
- Compute estimation method
- Operation counting
- Parameters
- 124,000,000
- Dataset size
- 200M
- Training hardware
- NVIDIA GeForce RTX 2080 Ti 11GB
- Chips used
- 1
- Training power draw
- 277 W
- Model accessibility
- Unreleased
- Open weights
- No
- Base model
- GPT-2 (124M)
- Citations
- 31
- Epoch confidence
- Speculative
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models