Pangu Pro MoE

Training compute

1.3×10²⁴ FLOP

Parameters

72B

Published

May 28, 2025

Pangu Pro MoE is an AI model developed by Huawei (China), first published in May 2025. It works in the language domain, on tasks such as language modeling/generation, question answering, quantitative reasoning and 2 more.

Training it took an estimated 1.3×10²⁴ FLOP of compute (estimation method: operation counting). The model has 71,990,000,000 parameters. It was trained on roughly 13T datapoints. Training ran on 4,000 Huawei Ascend 800T A2.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Huawei
Country of organization: China
Domain: Language
Task: Language modeling/generation, Question answering, Quantitative reasoning, Mathematical reasoning, Code generation
Training compute: 1.3×10²⁴ FLOP
Compute estimation method: Operation counting
Parameters: 71,990,000,000
Dataset size: 13T
Training hardware: Huawei Ascend 800T A2
Chips used: 4,000
Training power draw: 2.4 MW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Confident