ViT-Huge/14

Training compute

4.3×10²¹ FLOP

Parameters

632M

Published

Oct 22, 2020

ViT-Huge/14 is an AI model developed by Google Brain and Google Research (United States), first published in October 2020. It works in the vision domain, on tasks such as image representation.

Training it took an estimated 4.3×10²¹ FLOP of compute (estimation method: hardware). The model has 632,000,000 parameters. It was trained on roughly 303M datapoints. Training ran on Google TPU v3. The compute alone is estimated at $7K in 2023 dollars.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 62,651 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Brain, Google Research
Country of organization: United States
Domain: Vision
Task: Image representation
Training compute: 4.3×10²¹ FLOP
Compute estimation method: Hardware
Parameters: 632,000,000
Dataset size: 303M
Training hardware: Google TPU v3
Chip-hours: 30K
Training cost (2023 USD): $7K
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 62,651
Epoch confidence: Confident