GNMT

Training compute

6.6×10²¹ FLOP

Parameters

278M

Published

Sep 26, 2016

GNMT is an AI model developed by Google (United States), first published in September 2016. It works in the language domain, on tasks such as translation. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 6.6×10²¹ FLOP of compute (estimation method: hardware,third-party estimation). The model has 278,000,000 parameters. It was trained on roughly 720M datapoints. Training ran on NVIDIA Tesla K80. The compute alone is estimated at $201K in 2023 dollars.

Access: Hosted access (no API). Its weights are not openly released. The reference paper has 7,254 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Google
Country of organization: United States
Domain: Language
Task: Translation
Training compute: 6.6×10²¹ FLOP
Compute estimation method: Hardware, Third-party estimation
Parameters: 278,000,000
Dataset size: 720M
Training hardware: NVIDIA Tesla K80
Chip-hours: 655.7K
Training cost (2023 USD): $201K
Numerical format: FP32
Model accessibility: Hosted access (no API)
Open weights: No
Citations: 7,254
Epoch confidence: Likely