4 layer QRNN (h=2500)

Training compute

5.9×10¹⁷ FLOP

Parameters

151M

Published

Mar 22, 2018

4 layer QRNN (h=2500) is an AI model developed by Salesforce Research (United States), first published in March 2018. It works in the language domain, on tasks such as language modeling.

Training it took an estimated 5.9×10¹⁷ FLOP of compute (estimation method: hardware,operation counting). The model has 151,000,000 parameters. It was trained on roughly 103M datapoints. Training ran on 1 NVIDIA Quadro GP100 for about 12 hours.

Access: Unreleased. Its weights are not openly released. The reference paper has 183 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Salesforce Research
Country of organization: United States
Domain: Language
Task: Language modeling
Training compute: 5.9×10¹⁷ FLOP
Compute estimation method: Hardware, Operation counting
Parameters: 151,000,000
Dataset size: 103M
Training hardware: NVIDIA Quadro GP100
Chips used: 1
Training time: 12 h
Chip-hours: 12
Training power draw: 268 W
Numerical format: FP32
Model accessibility: Unreleased
Open weights: No
Citations: 183
Epoch confidence: Likely