YaLM

Training compute

2.2×10²³ FLOP

Parameters

100B

Published

Jun 23, 2022

YaLM is an AI model developed by Yandex (Russia), first published in June 2022. It works in the language domain, on tasks such as language modeling and chat.

Training it took an estimated 2.2×10²³ FLOP of compute (estimation method: hardware). The model has 100,000,000,000 parameters. It was trained on roughly 300B datapoints. Training ran on 800 NVIDIA A100 for about 1.6K hours.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Yandex
Country of organization: Russia
Domain: Language
Task: Language modeling, Chat
Training compute: 2.2×10²³ FLOP
Compute estimation method: Hardware
Parameters: 100,000,000,000
Dataset size: 300B
Training hardware: NVIDIA A100
Chips used: 800
Training time: 1,560 h
Chip-hours: 1.2M
Training power draw: 642.0 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Likely