AI models
ProBERTa
Training compute
9.7×10¹⁸ FLOP
Parameters
44M
Published
Sep 1, 2020
ProBERTa is an AI model developed by University of Illinois Urbana-Champaign (UIUC) and Reed College (United States), first published in September 2020. It works in the biology domain, on tasks such as proteins, protein representation learning, protein classification and protein interaction prediction.
Training it took an estimated 9.7×10¹⁸ FLOP of compute (estimation method: hardware). The model has 44,000,000 parameters. It was trained on roughly 58.3M datapoints. Training ran on 4 NVIDIA V100 for about 18 hours. The compute alone is estimated at $26 in 2023 dollars.
Access: Open weights (non-commercial). Its weights are openly available. The reference paper has 97 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- University of Illinois Urbana-Champaign (UIUC), Reed College
- Country of organization
- United States
- Domain
- Biology
- Task
- Proteins, Protein representation learning, Protein classification, Protein interaction prediction
- Training compute
- 9.7×10¹⁸ FLOP
- Compute estimation method
- Hardware
- Parameters
- 44,000,000
- Dataset size
- 58.3M
- Training hardware
- NVIDIA V100
- Chips used
- 4
- Training time
- 18 h
- Training power draw
- 2.4 kW
- Training cost (2023 USD)
- $26
- Numerical format
- FP16
- Model accessibility
- Open weights (non-commercial)
- Open weights
- Yes
- Citations
- 97
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models