GPT-3 175B (davinci)

Training compute

3.1×10²³ FLOP

Parameters

174.6B

Published

May 28, 2020

GPT-3 175B (davinci) is an AI model developed by OpenAI (United States), first published in May 2020. It works in the language domain, on tasks such as text autocompletion and language modeling/generation. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 3.1×10²³ FLOP of compute (estimation method: reported). The model has 174,600,000,000 parameters. It was trained on roughly 238B datapoints. Training ran on 10,000 NVIDIA Tesla V100 DGXS 32 GB for about 355 hours. The compute alone is estimated at $2M in 2023 dollars.

Access: API access. Its weights are not openly released. The reference paper has 58,240 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: OpenAI
Country of organization: United States
Domain: Language
Task: Text autocompletion, Language modeling/generation
Training compute: 3.1×10²³ FLOP
Compute estimation method: Reported
Parameters: 174,600,000,000
Dataset size: 238B
Training hardware: NVIDIA Tesla V100 DGXS 32 GB
Chips used: 10,000
Training time: 355 h
Chip-hours: 3.6M
Training power draw: 5.1 MW
Training cost (2023 USD): $2M
Model accessibility: API access
Open weights: No
Citations: 58,240
Epoch confidence: Confident