Transformer-XL + SIS

Training compute

3.8×10²⁰ FLOP

Parameters

246M

Published

May 3, 2021

Transformer-XL + SIS is an AI model developed by INRIA (France), first published in May 2021. It works in the language domain, on tasks such as language modeling.

Training it took an estimated 3.8×10²⁰ FLOP of compute (estimation method: comparison with other models). The model has 246,000,000 parameters.

Access: Unreleased. Its weights are not openly released. It is built on top of Transformer-XL (257M). The reference paper has 15 citations. Epoch AI rates the confidence of this record as speculative.

Full record

Organization: INRIA
Country of organization: France
Domain: Language
Task: Language modeling
Training compute: 3.8×10²⁰ FLOP
Compute estimation method: Comparison with other models
Parameters: 246,000,000
Model accessibility: Unreleased
Open weights: No
Base model: Transformer-XL (257M)
Citations: 15
Epoch confidence: Speculative