AlphaCode

Training compute

2.4×10²³ FLOP

Parameters

41.1B

Published

Feb 2, 2022

AlphaCode is an AI model developed by DeepMind (United Kingdom), first published in February 2022. It works in the language domain, on tasks such as code generation.

Training it took an estimated 2.4×10²³ FLOP of compute (estimation method: hardware). The model has 41,100,000,000 parameters. It was trained on roughly 967B datapoints. Training ran on 3,750 Google TPU v4 for about 147 hours.

Access: Unreleased. Its weights are not openly released. The reference paper has 2,162 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: DeepMind
Country of organization: United Kingdom
Domain: Language
Task: Code generation
Training compute: 2.4×10²³ FLOP
Compute estimation method: Hardware
Parameters: 41,100,000,000
Dataset size: 967B
Training hardware: Google TPU v4
Chips used: 3,750
Training time: 147 h
Training power draw: 2.6 MW
Numerical format: BF16
Model accessibility: Unreleased
Open weights: No
Citations: 2,162
Epoch confidence: Confident