AI models
GPT3-2.7B (FlashAttention-2)
Parameters
2.7B
Published
Jul 18, 2023
GPT3-2.7B (FlashAttention-2) is an AI model developed by Stanford University and Princeton University (United States), first published in July 2023. It works in the language domain, on tasks such as language modeling/generation.
Epoch AI has no training-compute estimate for this model. The model has 2,700,000,000 parameters. Training ran on 8 NVIDIA A100 SXM4 80 GB.
Access: Unreleased. Its weights are not openly released. The reference paper has 2,682 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Stanford University, Princeton University
- Country of organization
- United States
- Domain
- Language
- Task
- Language modeling/generation
- Parameters
- 2,700,000,000
- Training hardware
- NVIDIA A100 SXM4 80 GB
- Chips used
- 8
- Training power draw
- 6.4 kW
- Model accessibility
- Unreleased
- Open weights
- No
- Citations
- 2,682
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models