Local Transformer (WT103)

Published

Mar 12, 2020

Local Transformer (WT103) is an AI model developed by Google Research (United States), first published in March 2020. It works in the language domain, on tasks such as language modeling.

Epoch AI has no training-compute estimate for this model. It was trained on roughly 103M datapoints. Training ran on 64 Google TPU v3.

Access: Unreleased. Its weights are not openly released. The reference paper has 754 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Research
Country of organization: United States
Domain: Language
Task: Language modeling
Dataset size: 103M
Training hardware: Google TPU v3
Chips used: 64
Training power draw: 58.9 kW
Model accessibility: Unreleased
Open weights: No
Citations: 754
Epoch confidence: Confident