BERT-Large-CAS (WT103)

Parameters

340M

Published

Apr 20, 2019

BERT-Large-CAS (WT103) is an AI model developed by Amazon (United States), first published in April 2019. It works in the language domain, on tasks such as neural architecture search - nas and language modeling.

Epoch AI has no training-compute estimate for this model. The model has 340,000,000 parameters. It was trained on roughly 103M datapoints.

Access: Unreleased. Its weights are not openly released. It is built on top of BERT-Large. The reference paper has 139 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Amazon
Country of organization: United States
Domain: Language
Task: Neural Architecture Search - NAS, Language modeling
Parameters: 340,000,000
Dataset size: 103M
Model accessibility: Unreleased
Open weights: No
Base model: BERT-Large
Citations: 139
Epoch confidence: Confident