data2vec (language)

Parameters

705.1M

Published

Jan 20, 2022

data2vec (language) is an AI model developed by Meta AI (United States), first published in January 2022. It works in the language domain, on tasks such as language modeling/generation.

Epoch AI has no training-compute estimate for this model. The model has 705,134,592 parameters. It was trained on roughly 131.1B datapoints.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 1,021 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Meta AI
Country of organization: United States
Domain: Language
Task: Language modeling/generation
Parameters: 705,134,592
Dataset size: 131.1B
Numerical format: FP16
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 1,021
Epoch confidence: Likely