HyenaDNA
HyenaDNA is an AI model developed by Stanford University, Harvard University, Mila - Quebec AI (originally Montreal Institute for Learning Algorithms) and University of Montreal / Université de Montréal (United States and Canada), first published in June 2023. It works in the biology domain, on tasks such as protein or nucleotide language model (plm/nlm).
Training it took an estimated 1.8×10²¹ FLOP of compute (estimation method: hardware). The model has 6,600,000 parameters. It was trained on roughly 2.9B datapoints. Training ran on 8 NVIDIA A100 for about 672 hours. The compute alone is estimated at $5K in 2023 dollars.
Access: Open weights (restricted use). Its weights are openly available. The reference paper has 477 citations. Epoch AI rates the confidence of this record as confident.
- Organization
- Stanford University, Harvard University, Mila - Quebec AI (originally Montreal Institute for Learning Algorithms), University of Montreal / Université de Montréal
- Country of organization
- United States, Canada
- Domain
- Biology
- Task
- Protein or nucleotide language model (pLM/nLM)
- Training compute
- 1.8×10²¹ FLOP
- Compute estimation method
- Hardware
- Parameters
- 6,600,000
- Dataset size
- 2.9B
- Training hardware
- NVIDIA A100
- Chips used
- 8
- Training time
- 672 h
- Training power draw
- 6.4 kW
- Training cost (2023 USD)
- $5K
- Numerical format
- FP16
- Model accessibility
- Open weights (restricted use)
- Open weights
- Yes
- Citations
- 477
- Epoch confidence
- Confident