GPT-Neo-2.7B

Training compute

7.9×10²¹ FLOP

Parameters

2.7B

Published

Mar 21, 2021

GPT-Neo-2.7B is an AI model developed by EleutherAI (United States), first published in March 2021. It works in the language domain, on tasks such as language modeling/generation and question answering.

Training it took an estimated 7.9×10²¹ FLOP of compute (estimation method: third-party estimation,operation counting). The model has 2,700,000,000 parameters. It was trained on roughly 420B datapoints.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 880 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: EleutherAI
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Question answering
Training compute: 7.9×10²¹ FLOP
Compute estimation method: Third-party estimation, Operation counting
Parameters: 2,700,000,000
Dataset size: 420B
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 880
Epoch confidence: Confident

Benchmark results

01	PIQA	Score	0.73
02	BoolQ	Score	0.62
03	HellaSwag	Overall accuracy	0.43
04	OpenBookQA	Accuracy	0.23