AI models
RoBERTa Base
Training compute
1.5×10²¹ FLOP
Parameters
125M
Published
Jul 1, 2019
RoBERTa Base is an AI model developed by Facebook and University of Washington (United States), first published in July 2019. It works in the language domain, on tasks such as question answering and language modeling/generation.
Training it took an estimated 1.5×10²¹ FLOP of compute (estimation method: operation counting). The model has 125,000,000 parameters.
Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 29,641 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Facebook, University of Washington
- Country of organization
- United States
- Domain
- Language
- Task
- Question answering, Language modeling/generation
- Training compute
- 1.5×10²¹ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 125,000,000
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Citations
- 29,641
- Epoch confidence
- Confident
More from Facebook,University of Washington
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models