GPT-3.5 (davinci-002)

Training compute

2.6×10²⁴ FLOP

Published

Mar 15, 2022

GPT-3.5 (davinci-002) is an AI model developed by OpenAI (United States), first published in March 2022. It works in the language domain, on tasks such as language modeling. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 2.6×10²⁴ FLOP of compute (estimation method: comparison with other models,benchmarks). Training ran on NVIDIA A100 SXM4 40 GB. The compute alone is estimated at $5M in 2023 dollars.

Access: API access. Its weights are not openly released. Epoch AI rates the confidence of this record as speculative.

Full record

Organization: OpenAI
Country of organization: United States
Domain: Language
Task: Language modeling
Training compute: 2.6×10²⁴ FLOP
Compute estimation method: Comparison with other models, Benchmarks
Training hardware: NVIDIA A100 SXM4 40 GB
Training cost (2023 USD): $5M
Model accessibility: API access
Open weights: No
Epoch confidence: Speculative