MSA Transformer
MSA Transformer is an AI model developed by Facebook AI Research, University of California (UC) Berkeley and New York University (NYU) (United States and France), first published in February 2021. It works in the biology domain, on tasks such as proteins, protein or nucleotide language model (plm/nlm), protein contact and distance prediction and protein folding prediction.
Training it took an estimated 5.5×10²¹ FLOP of compute (estimation method: operation counting). The model has 100,000,000 parameters. It was trained on roughly 1.4T datapoints. Training ran on 32 NVIDIA Tesla V100 DGXS 32 GB. The compute alone is estimated at $13K in 2023 dollars.
Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 644 citations. Epoch AI rates the confidence of this record as likely.
- Organization
- Facebook AI Research, University of California (UC) Berkeley, New York University (NYU)
- Country of organization
- United States, France
- Domain
- Biology
- Task
- Proteins, Protein or nucleotide language model (pLM/nLM), Protein contact and distance prediction, Protein folding prediction
- Training compute
- 5.5×10²¹ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 100,000,000
- Dataset size
- 1.4T
- Training hardware
- NVIDIA Tesla V100 DGXS 32 GB
- Chips used
- 32
- Training power draw
- 16.2 kW
- Training cost (2023 USD)
- $13K
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Citations
- 644
- Epoch confidence
- Likely