Primer (GPT-3 XL-like 1.9B)

Training compute

2.2×10²² FLOP

Parameters

1.9B

Published

Jan 24, 2022

Primer (GPT-3 XL-like 1.9B) is an AI model developed by Google Brain (United States), first published in January 2022. It works in the language domain, on tasks such as language modeling/generation.

Training it took an estimated 2.2×10²² FLOP of compute (estimation method: hardware,operation counting). The model has 1,900,000,000 parameters. It was trained on roughly 2T datapoints. Training ran on 512 Google TPU v4 for about 140 hours.

Access: Unreleased. Its weights are not openly released. The reference paper has 209 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Brain
Country of organization: United States
Domain: Language
Task: Language modeling/generation
Training compute: 2.2×10²² FLOP
Compute estimation method: Hardware, Operation counting
Parameters: 1,900,000,000
Dataset size: 2T
Training hardware: Google TPU v4
Chips used: 512
Training time: 140 h
Training power draw: 350.4 kW
Model accessibility: Unreleased
Open weights: No
Citations: 209
Epoch confidence: Confident