OPT-IML (175B)

Training compute

4.3×10²³ FLOP

Parameters

175B

Published

Dec 22, 2022

OPT-IML (175B) is an AI model developed by Meta AI (United States), first published in December 2022. It works in the language domain, on tasks such as language modeling.

Training it took an estimated 4.3×10²³ FLOP of compute (estimation method: operation counting). The model has 175,000,000,000 parameters. It was trained on roughly 2B datapoints. Training ran on 128 NVIDIA A100 for about 72 hours.

Access: Open weights (non-commercial). Its weights are openly available. It is built on top of OPT-175B. The reference paper has 304 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Meta AI
Country of organization: United States
Domain: Language
Task: Language modeling
Training compute: 4.3×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 175,000,000,000
Dataset size: 2B
Training hardware: NVIDIA A100
Chips used: 128
Training time: 72 h
Chip-hours: 9.2K
Training power draw: 102.3 kW
Model accessibility: Open weights (non-commercial)
Open weights: Yes
Base model: OPT-175B
Citations: 304
Epoch confidence: Likely