AraBERT LArge v2

Training compute

1.5×10²¹ FLOP

Parameters

371M

Published

Mar 30, 2020

AraBERT LArge v2 is an AI model developed by American University of Beirut (Lebanon), first published in March 2020. It works in the language domain, on tasks such as language modeling/generation and question answering.

Training it took an estimated 1.5×10²¹ FLOP of compute (estimation method: operation counting,hardware). The model has 371,000,000 parameters. Training ran on 128 Google TPU v3 for about 168 hours.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: American University of Beirut
Country of organization: Lebanon
Domain: Language
Task: Language modeling/generation, Question answering
Training compute: 1.5×10²¹ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 371,000,000
Training hardware: Google TPU v3
Chips used: 128
Training time: 168 h
Training power draw: 117.7 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Confident