T5-3B

Training compute

9×10²¹ FLOP

Parameters

2.8B

Published

Oct 23, 2019

T5-3B is an AI model developed by Google (United States), first published in October 2019. It works in the language domain, on tasks such as text autocompletion. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 9×10²¹ FLOP of compute (estimation method: third-party estimation,reported). The model has 2,800,000,000 parameters. It was trained on roughly 5.1B datapoints. Training ran on Google TPU v3. The compute alone is estimated at $17K in 2023 dollars.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 25,683 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google
Country of organization: United States
Domain: Language
Task: Text autocompletion
Training compute: 9×10²¹ FLOP
Compute estimation method: Third-party estimation, Reported
Parameters: 2,800,000,000
Dataset size: 5.1B
Training hardware: Google TPU v3
Training cost (2023 USD): $17K
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 25,683
Epoch confidence: Confident

Benchmark results

01	BoolQ	Score	0.9
02	SuperGLUE	Score	0.86
03	CommonsenseQA 2	Score	0.6