Polyglot-Ko-12.8B

Training compute

1.3×10²² FLOP

Parameters

12.9B

Published

Jun 4, 2023

Polyglot-Ko-12.8B is an AI model developed by EleutherAI (United States), first published in June 2023. It works in the language domain, on tasks such as translation and language modeling/generation.

Training it took an estimated 1.3×10²² FLOP of compute (estimation method: operation counting). The model has 12,898,631,680 parameters. It was trained on roughly 95.8B datapoints. Training ran on 256 NVIDIA A100.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 35 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: EleutherAI
Country of organization: United States
Domain: Language
Task: Translation, Language modeling/generation
Training compute: 1.3×10²² FLOP
Compute estimation method: Operation counting
Parameters: 12,898,631,680
Dataset size: 95.8B
Training hardware: NVIDIA A100
Chips used: 256
Training power draw: 203.9 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 35
Epoch confidence: Confident