Transformer-XL + FWL

Parameters

257M

Published

Dec 5, 2022

Transformer-XL + FWL is an AI model developed by Google Research (United States), first published in December 2022. It works in the language domain, on tasks such as language modeling/generation.

Epoch AI has no training-compute estimate for this model. The model has 257,000,000 parameters. It was trained on roughly 103M datapoints.

Access: Unreleased. Its weights are not openly released. The reference paper has 19 citations.

Full record

Organization: Google Research
Country of organization: United States
Domain: Language
Task: Language modeling/generation
Parameters: 257,000,000
Dataset size: 103M
Model accessibility: Unreleased
Open weights: No
Citations: 19