Flamingo

Training compute

2.2×10²³ FLOP

Parameters

80B

Published

Apr 29, 2022

Flamingo is an AI model developed by DeepMind (United Kingdom), first published in April 2022. It works in the multimodal, vision, language and video domain, on tasks such as visual question answering and image captioning.

Training it took an estimated 2.2×10²³ FLOP of compute (estimation method: hardware). The model has 80,000,000,000 parameters. It was trained on roughly 458.3B datapoints. Training ran on 1,536 Google TPU v4 for about 360 hours. The compute alone is estimated at $183K in 2023 dollars.

Access: Unreleased. Its weights are not openly released. It is built on top of Chinchilla. The reference paper has 5,793 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: DeepMind
Country of organization: United Kingdom
Domain: Multimodal, Vision, Language, Video
Task: Visual question answering, Image captioning
Training compute: 2.2×10²³ FLOP
Compute estimation method: Hardware
Parameters: 80,000,000,000
Dataset size: 458.3B
Training hardware: Google TPU v4
Chips used: 1,536
Training time: 360 h
Chip-hours: 553K
Training power draw: 1.0 MW
Training cost (2023 USD): $183K
Numerical format: BF16
Model accessibility: Unreleased
Open weights: No
Base model: Chinchilla
Citations: 5,793
Epoch confidence: Confident