PanGu-Σ

Training compute

4.7×10²³ FLOP

Parameters

1.1T

Published

Mar 20, 2023

PanGu-Σ is an AI model developed by Huawei Noah's Ark Lab (China), first published in March 2023. It works in the language domain, on tasks such as code generation, language modeling, translation and question answering.

Training it took an estimated 4.7×10²³ FLOP of compute (estimation method: hardware). The model has 1,085,000,000,000 parameters. It was trained on roughly 329B datapoints. Training ran on 512 Huawei Ascend 910 for about 2.4K hours.

Access: Unreleased. Its weights are not openly released. The reference paper has 78 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Huawei Noah's Ark Lab
Country of organization: China
Domain: Language
Task: Code generation, Language modeling, Translation, Question answering
Training compute: 4.7×10²³ FLOP
Compute estimation method: Hardware
Parameters: 1,085,000,000,000
Dataset size: 329B
Training hardware: Huawei Ascend 910
Chips used: 512
Training time: 2,400 h
Chip-hours: 1.2M
Training power draw: 316.5 kW
Numerical format: FP16
Model accessibility: Unreleased
Open weights: No
Citations: 78
Epoch confidence: Confident