Pythia-410m
Pythia-410m is an AI model developed by EleutherAI, Booz Allen Hamilton, McLean, University of Cambridge, Indraprastha Institute of Information Technology Delhi, Stability AI and 2 more (United States, United Kingdom, India and Netherlands), first published in April 2023. It works in the language domain, on tasks such as language modeling/generation and question answering.
Training it took an estimated 7.4×10²⁰ FLOP of compute (estimation method: operation counting). The model has 410,000,000 parameters. It was trained on roughly 299.9B datapoints. Training ran on 32 NVIDIA A100 SXM4 40 GB.
Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 1,862 citations. Epoch AI rates the confidence of this record as confident.
- Organization
- EleutherAI, Booz Allen Hamilton, McLean, University of Cambridge, Indraprastha Institute of Information Technology Delhi, Stability AI, datasaur.ai, University of Amsterdam
- Country of organization
- United States, United Kingdom, India, Netherlands
- Domain
- Language
- Task
- Language modeling/generation, Question answering
- Training compute
- 7.4×10²⁰ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 410,000,000
- Dataset size
- 299.9B
- Training hardware
- NVIDIA A100 SXM4 40 GB
- Chips used
- 32
- Chip-hours
- 2.5K
- Training power draw
- 25.5 kW
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Citations
- 1,862
- Epoch confidence
- Confident