Gemini 1.0 Ultra

Training compute

5×10²⁵ FLOP

Published

Dec 6, 2023

Gemini 1.0 Ultra is an AI model developed by Google DeepMind (United States), first published in December 2023. It works in the multimodal, language and vision domain, on tasks such as language modeling, visual question answering, chat and translation. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 5×10²⁵ FLOP of compute (estimation method: benchmarks,hardware). Training ran on 57,000 Google TPU v4 for about 2.4K hours. The compute alone is estimated at $31M in 2023 dollars.

Access: API access. Its weights are not openly released. The reference paper has 633 citations. Epoch AI rates the confidence of this record as speculative.

Full record

Organization: Google DeepMind
Country of organization: United States
Domain: Multimodal, Language, Vision
Task: Language modeling, Visual question answering, Chat, Translation
Training compute: 5×10²⁵ FLOP
Compute estimation method: Benchmarks, Hardware
Training hardware: Google TPU v4
Chips used: 57,000
Training time: 2,400 h
Chip-hours: 132M
Training power draw: 38.4 MW
Training cost (2023 USD): $31M
Model accessibility: API access
Open weights: No
Citations: 633
Epoch confidence: Speculative