PLATO-XL

Training compute

9.9×10²¹ FLOP

Parameters

11B

Published

Sep 20, 2021

PLATO-XL is an AI model developed by Baidu (China), first published in September 2021. It works in the language domain, on tasks such as language modeling.

Training it took an estimated 9.9×10²¹ FLOP of compute (estimation method: operation counting). The model has 11,000,000,000 parameters. It was trained on roughly 150B datapoints. Training ran on 256 NVIDIA V100.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 69 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Baidu
Country of organization: China
Domain: Language
Task: Language modeling
Training compute: 9.9×10²¹ FLOP
Compute estimation method: Operation counting
Parameters: 11,000,000,000
Dataset size: 150B
Training hardware: NVIDIA V100
Chips used: 256
Training power draw: 155.0 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 69
Epoch confidence: Confident