FLAN 137B

Training compute

2×10²⁴ FLOP

Parameters

137B

Published

Sep 3, 2021

FLAN 137B is an AI model developed by Google Research (United States), first published in September 2021. It works in the language domain, on tasks such as language modeling, question answering and language modeling/generation. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 2×10²⁴ FLOP of compute (estimation method: operation counting). The model has 137,000,000,000 parameters. It was trained on roughly 2.5T datapoints. Training ran on Google TPU v3.

Access: Unreleased. Its weights are not openly released. It is built on top of LaMDA. The reference paper has 5,011 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Research
Country of organization: United States
Domain: Language
Task: Language modeling, Question answering, Language modeling/generation
Training compute: 2×10²⁴ FLOP
Compute estimation method: Operation counting
Parameters: 137,000,000,000
Dataset size: 2.5T
Training hardware: Google TPU v3
Chip-hours: 3.8K
Model accessibility: Unreleased
Open weights: No
Base model: LaMDA
Citations: 5,011
Epoch confidence: Confident