AI models
GPT-J-6B
Training compute
1.5×10²² FLOP
Parameters
6.1B
Published
May 1, 2021
GPT-J-6B is an AI model developed by EleutherAI and LAION (United States and Germany), first published in May 2021. It works in the language domain, on tasks such as language modeling/generation, question answering, automated theorem proving and code generation.
Training it took an estimated 1.5×10²² FLOP of compute (estimation method: reported). The model has 6,053,381,344 parameters. It was trained on roughly 400B datapoints. Training ran on Google TPU v3 for about 840 hours.
Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- EleutherAI, LAION
- Country of organization
- United States, Germany
- Domain
- Language
- Task
- Language modeling/generation, Question answering, Automated theorem proving, Code generation
- Training compute
- 1.5×10²² FLOP
- Compute estimation method
- Reported
- Parameters
- 6,053,381,344
- Dataset size
- 400B
- Training hardware
- Google TPU v3
- Training time
- 840 h
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Epoch confidence
- Confident
Benchmark results
| 01 | PIQA | Score | 0.75 |
| 02 | HellaSwag | Overall accuracy | 0.66 |
| 03 | BoolQ | Score | 0.65 |
| 04 | WinoGrande | Accuracy | 0.65 |
| 05 | OpenBookQA | Accuracy | 0.38 |
| 06 | ARC AI2 | Challenge score | 0.36 |
| 07 | MMLU | EM | 0.26 |
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models