Transformer-XL DeFINE (107M)

Parameters

107.4M

Published

Nov 27, 2019

Transformer-XL DeFINE (107M) is an AI model developed by University of Washington (United States), first published in November 2019. It works in the language domain, on tasks such as language modeling and translation.

Epoch AI has no training-compute estimate for this model. The model has 107,370,000 parameters. Training ran on 4 NVIDIA GeForce RTX 2080 Ti 11GB.

Access: Unreleased. Its weights are not openly released. The reference paper has 29 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: University of Washington
Country of organization: United States
Domain: Language
Task: Language modeling, Translation
Parameters: 107,370,000
Training hardware: NVIDIA GeForce RTX 2080 Ti 11GB
Chips used: 4
Training power draw: 2.0 kW
Model accessibility: Unreleased
Open weights: No
Citations: 29
Epoch confidence: Confident