Nucleotide Transformer
Nucleotide Transformer is an AI model developed by NVIDIA, Technical University of Munich and InstaDeep (United States, Germany and United Kingdom), first published in January 2023. It works in the biology domain, on tasks such as protein or nucleotide language model (plm/nlm) and nucleotide generation.
Training it took an estimated 8.1×10²¹ FLOP of compute (estimation method: operation counting,hardware). The model has 2,500,000,000 parameters. It was trained on roughly 300B datapoints. Training ran on 128 NVIDIA A100 for about 672 hours. The compute alone is estimated at $51K in 2023 dollars.
Access: Open weights (non-commercial). Its weights are openly available. The reference paper has 22 citations. Epoch AI rates the confidence of this record as likely.
- Organization
- NVIDIA, Technical University of Munich, InstaDeep
- Country of organization
- United States, Germany, United Kingdom
- Domain
- Biology
- Task
- Protein or nucleotide language model (pLM/nLM), Nucleotide generation
- Training compute
- 8.1×10²¹ FLOP
- Compute estimation method
- Operation counting, Hardware
- Parameters
- 2,500,000,000
- Dataset size
- 300B
- Training hardware
- NVIDIA A100
- Chips used
- 128
- Training time
- 672 h
- Chip-hours
- 86K
- Training power draw
- 102.2 kW
- Training cost (2023 USD)
- $51K
- Numerical format
- FP32
- Model accessibility
- Open weights (non-commercial)
- Open weights
- Yes
- Citations
- 22
- Epoch confidence
- Likely