DistBelief NNLM

Training compute

2.6×10¹⁸ FLOP

Published

Jan 16, 2013

DistBelief NNLM is an AI model developed by Google (United States), first published in January 2013. It works in the language domain, on tasks such as semantic embedding. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 2.6×10¹⁸ FLOP of compute (estimation method: hardware). It was trained on roughly 6B datapoints. The compute alone is estimated at $3K in 2023 dollars.

The reference paper has 41,000 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Google
Country of organization: United States
Domain: Language
Task: Semantic embedding
Training compute: 2.6×10¹⁸ FLOP
Compute estimation method: Hardware
Dataset size: 6B
Chips used: 180
Training time: 336 h
Training cost (2023 USD): $3K
Citations: 41,000
Epoch confidence: Likely