Knowledge distillation student model

Training compute

10¹⁷ FLOP

Parameters

84M

Published

Sep 22, 2016

Knowledge distillation student model is an AI model developed by Harvard University (United States), first published in September 2016. It works in the language domain, on tasks such as translation.

Training it took an estimated 10¹⁷ FLOP of compute (estimation method: operation counting). The model has 84,000,000 parameters. It was trained on roughly 100M datapoints.

Epoch AI rates the confidence of this record as speculative.

Full record

Organization: Harvard University
Country of organization: United States
Domain: Language
Task: Translation
Training compute: 10¹⁷ FLOP
Compute estimation method: Operation counting
Parameters: 84,000,000
Dataset size: 100M
Epoch confidence: Speculative