AraBERT

Training compute

3.2×10¹⁹ FLOP

Parameters

110M

Published

Mar 30, 2020

AraBERT is an AI model developed by American University of Beirut (Lebanon), first published in March 2020. It works in the language domain, on tasks such as language modeling/generation and question answering.

Training it took an estimated 3.2×10¹⁹ FLOP of compute (estimation method: operation counting,hardware). The model has 110,000,000 parameters. Training ran on 8 Google TPU v2 for about 96 hours.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: American University of Beirut
Country of organization: Lebanon
Domain: Language
Task: Language modeling/generation, Question answering
Training compute: 3.2×10¹⁹ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 110,000,000
Training hardware: Google TPU v2
Chips used: 8
Training time: 96 h
Training power draw: 4.6 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Confident