M6-T

Training compute

5.5×10²¹ FLOP

Parameters

1T

Published

Mar 5, 2021

M6-T is an AI model developed by Alibaba (China), first published in March 2021. It works in the multimodal, language and vision domain, on tasks such as chat and image captioning.

Training it took an estimated 5.5×10²¹ FLOP of compute (estimation method: third-party estimation). The model has 1,002,700,000,000 parameters. It was trained on roughly 111.8B datapoints. Training ran on 480 NVIDIA Tesla V100 DGXS 32 GB. The compute alone is estimated at $13K in 2023 dollars.

Access: Unreleased. Its weights are not openly released. The reference paper has 76 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Alibaba
Country of organization: China
Domain: Multimodal, Language, Vision
Task: Chat, Image captioning
Training compute: 5.5×10²¹ FLOP
Compute estimation method: Third-party estimation
Parameters: 1,002,700,000,000
Dataset size: 111.8B
Training hardware: NVIDIA Tesla V100 DGXS 32 GB
Chips used: 480
Training power draw: 243.3 kW
Training cost (2023 USD): $13K
Numerical format: FP16
Model accessibility: Unreleased
Open weights: No
Citations: 76
Epoch confidence: Likely