GPT2-Large+LHOPT

Training compute

5×10²¹ FLOP

Parameters

760M

Published

Jun 2, 2021

GPT2-Large+LHOPT is an AI model developed by OpenAI (United States), first published in June 2021. It works in the language domain, on tasks such as language modeling/generation.

Training it took an estimated 5×10²¹ FLOP of compute (estimation method: operation counting). The model has 760,000,000 parameters. It was trained on roughly 103M datapoints.

Access: Unreleased. Its weights are not openly released. It is built on top of GPT-2 (774M). The reference paper has 35 citations. Epoch AI rates the confidence of this record as speculative.

Full record

Organization: OpenAI
Country of organization: United States
Domain: Language
Task: Language modeling/generation
Training compute: 5×10²¹ FLOP
Compute estimation method: Operation counting
Parameters: 760,000,000
Dataset size: 103M
Model accessibility: Unreleased
Open weights: No
Base model: GPT-2 (774M)
Citations: 35
Epoch confidence: Speculative