JIUTIAN-139MoE

Training compute

4.4×10²³ FLOP

Parameters

38.8B

Published

Jun 15, 2024

JIUTIAN-139MoE is an AI model developed by China Mobile (China), first published in June 2024. It works in the language domain, on tasks such as language modeling/generation, question answering and code generation.

Training it took an estimated 4.4×10²³ FLOP of compute (estimation method: operation counting,hardware). The model has 38,800,000,000 parameters. Training ran on 896 NVIDIA A800 PCIe,Ascend (昇腾) 910B for about 1.5K hours.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: China Mobile
Country of organization: China
Domain: Language
Task: Language modeling/generation, Question answering, Code generation
Training compute: 4.4×10²³ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 38,800,000,000
Training hardware: NVIDIA A800 PCIe, Ascend (昇腾) 910B
Chips used: 896
Training time: 1,464 h
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Confident