ERNIE-GEN (large)

Training compute

2×10²⁰ FLOP

Parameters

340M

Published

Aug 6, 2020

ERNIE-GEN (large) is an AI model developed by Baidu (China), first published in August 2020. It works in the language domain, on tasks such as language modeling, language modeling/generation, text summarization and chat.

Training it took an estimated 2×10²⁰ FLOP of compute (estimation method: operation counting). The model has 340,000,000 parameters. It was trained on roughly 114.7B datapoints.

Access: Open weights (non-commercial). Its weights are openly available. The reference paper has 133 citations. Epoch AI rates the confidence of this record as speculative.

Full record

Organization: Baidu
Country of organization: China
Domain: Language
Task: Language modeling, Language modeling/generation, Text summarization, Chat
Training compute: 2×10²⁰ FLOP
Compute estimation method: Operation counting
Parameters: 340,000,000
Dataset size: 114.7B
Numerical format: FP16
Model accessibility: Open weights (non-commercial)
Open weights: Yes
Citations: 133
Epoch confidence: Speculative