ERNIE 3.0 Titan
ERNIE 3.0 Titan is an AI model developed by Baidu and Peng Cheng Laboratory (China), first published in December 2021. It works in the language domain, on tasks such as language modeling, language modeling/generation, relation extraction and 2 more. It counts among the frontier models: the systems trained with the most compute of their moment.
Training it took an estimated 10²⁴ FLOP of compute (estimation method: operation counting). The model has 260,000,000,000 parameters. It was trained on roughly 668B datapoints. Training ran on 1,920 NVIDIA Tesla V100 DGXS 32 GB,Huawei Ascend 910.
Access: Hosted access (no API). Its weights are not openly released. The reference paper has 87 citations. Epoch AI rates the confidence of this record as confident.
- Organization
- Baidu, Peng Cheng Laboratory
- Country of organization
- China
- Domain
- Language
- Task
- Language modeling, Language modeling/generation, Relation extraction, Sentiment classification, Text classification
- Training compute
- 10²⁴ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 260,000,000,000
- Dataset size
- 668B
- Training hardware
- NVIDIA Tesla V100 DGXS 32 GB, Huawei Ascend 910
- Chips used
- 1,920
- Numerical format
- FP16
- Model accessibility
- Hosted access (no API)
- Open weights
- No
- Citations
- 87
- Epoch confidence
- Confident