PanGu-α

Training compute

5.1×10²² FLOP

Parameters

207B

Published

Apr 25, 2021

PanGu-α is an AI model developed by Huawei Noah's Ark Lab (China), first published in April 2021. It works in the language domain, on tasks such as language modeling/generation, text summarization and question answering.

Training it took an estimated 5.1×10²² FLOP of compute (estimation method: operation counting,third-party estimation). The model has 207,000,000,000 parameters. Training ran on 2,048 Huawei Ascend 910.

Access: Unreleased. Its weights are not openly released. The reference paper has 234 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Huawei Noah's Ark Lab
Country of organization: China
Domain: Language
Task: Language modeling/generation, Text summarization, Question answering
Training compute: 5.1×10²² FLOP
Compute estimation method: Operation counting, Third-party estimation
Parameters: 207,000,000,000
Training hardware: Huawei Ascend 910
Chips used: 2,048
Training power draw: 1.3 MW
Model accessibility: Unreleased
Open weights: No
Citations: 234
Epoch confidence: Likely