ALIGN

Training compute

2.6×10²² FLOP

Parameters

820M

Published

Jun 11, 2021

ALIGN is an AI model developed by Google Research (United States), first published in June 2021. It works in the multimodal, vision and language domain, on tasks such as representation learning, image classification and image representation.

Training it took an estimated 2.6×10²² FLOP of compute (estimation method: hardware). The model has 820,000,000 parameters. It was trained on roughly 1.8B datapoints. Training ran on 512 Google TPU v3 for about 347 hours. The compute alone is estimated at $33K in 2023 dollars.

Access: Unreleased. Its weights are not openly released. The reference paper has 5,463 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Research
Country of organization: United States
Domain: Multimodal, Vision, Language
Task: Representation learning, Image classification, Image representation
Training compute: 2.6×10²² FLOP
Compute estimation method: Hardware
Parameters: 820,000,000
Dataset size: 1.8B
Training hardware: Google TPU v3
Chips used: 512
Training time: 347 h
Chip-hours: 177.8K
Training power draw: 466.1 kW
Training cost (2023 USD): $33K
Model accessibility: Unreleased
Open weights: No
Citations: 5,463
Epoch confidence: Confident