RNN for 1B words

Parameters

20B

Published

Dec 11, 2013

RNN for 1B words is an AI model developed by Google (United States), first published in December 2013. It works in the language domain, on tasks such as language modeling.

Epoch AI has no training-compute estimate for this model. The model has 20,000,000,000 parameters. It was trained on roughly 1B datapoints.

The reference paper has 1,205 citations. Epoch AI rates the confidence of this record as speculative.

Full record

Organization: Google
Country of organization: United States
Domain: Language
Task: Language modeling
Parameters: 20,000,000,000
Dataset size: 1B
Chips used: 24
Training time: 240 h
Chip-hours: 5.8K
Citations: 1,205
Epoch confidence: Speculative