AI models
Transformer+Recurrent Windows of Context
Training compute
7.9×10²⁰ FLOP
Parameters
124M
Published
Aug 16, 2020
Transformer+Recurrent Windows of Context is an AI model developed by Toyota Technological Institute at Chicago and University of Chicago (United States), first published in August 2020. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 7.9×10²⁰ FLOP of compute (estimation method: operation counting). The model has 124,000,000 parameters.
Access: Unreleased. Its weights are not openly released. It is built on top of GPT-2 (124M). The reference paper has 7 citations. Epoch AI rates the confidence of this record as speculative.
Full record
- Organization
- Toyota Technological Institute at Chicago, University of Chicago
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling
- Training compute
- 7.9×10²⁰ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 124,000,000
- Model accessibility
- Unreleased
- Open weights
- No
- Base model
- GPT-2 (124M)
- Citations
- 7
- Epoch confidence
- Speculative
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models