Flan-T5 11B

Training compute

3.3×10²² FLOP

Parameters

11B

Published

Oct 20, 2022

Flan-T5 11B is an AI model developed by Google (United States), first published in October 2022. It works in the language domain, on tasks such as language modeling/generation.

Training it took an estimated 3.3×10²² FLOP of compute (estimation method: reported). The model has 11,000,000,000 parameters. It was trained on roughly 100B datapoints. Training ran on Google TPU v4. The compute alone is estimated at $98K in 2023 dollars.

Access: Open weights (unrestricted). Its weights are openly available. It is built on top of T5-11B. The reference paper has 4,129 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google
Country of organization: United States
Domain: Language
Task: Language modeling/generation
Training compute: 3.3×10²² FLOP
Compute estimation method: Reported
Parameters: 11,000,000,000
Dataset size: 100B
Training hardware: Google TPU v4
Training cost (2023 USD): $98K
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Base model: T5-11B
Citations: 4,129
Epoch confidence: Confident