U-PaLM (540B)
U-PaLM (540B) is an AI model developed by Google (United States), first published in October 2022. It works in the language domain, on tasks such as language generation, language modeling/generation, question answering and 2 more. It counts among the frontier models: the systems trained with the most compute of their moment.
Training it took an estimated 2.5×10²⁴ FLOP of compute (estimation method: comparison with other models). The model has 540,000,000,000 parameters. It was trained on roughly 1.3B datapoints. Training ran on 512 Google TPU v4 for about 120 hours.
Access: Unreleased. Its weights are not openly released. It is built on top of PaLM (540B). The reference paper has 76 citations. Epoch AI rates the confidence of this record as confident.
- Organization
- Country of organization
- United States
- Domain
- Language
- Task
- Language generation, Language modeling/generation, Question answering, Mathematical reasoning, Quantitative reasoning
- Training compute
- 2.5×10²⁴ FLOP
- Compute estimation method
- Comparison with other models
- Parameters
- 540,000,000,000
- Dataset size
- 1.3B
- Training hardware
- Google TPU v4
- Chips used
- 512
- Training time
- 120 h
- Chip-hours
- 61.4K
- Training power draw
- 348.3 kW
- Model accessibility
- Unreleased
- Open weights
- No
- Base model
- PaLM (540B)
- Citations
- 76
- Epoch confidence
- Confident