CoCa

Training compute

7.3×10²² FLOP

Parameters

2.1B

Published

Jun 14, 2022

CoCa is an AI model developed by Google Research (United States), first published in June 2022. It works in the vision domain, on tasks such as image classification, visual question answering and image captioning.

Training it took an estimated 7.3×10²² FLOP of compute (estimation method: hardware). The model has 2,100,000,000 parameters. It was trained on roughly 1.4T datapoints. Training ran on 2,048 Google TPU v4 for about 120 hours. The compute alone is estimated at $78K in 2023 dollars.

Access: Unreleased. Its weights are not openly released. The reference paper has 1,719 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Research
Country of organization: United States
Domain: Vision
Task: Image classification, Visual question answering, Image captioning
Training compute: 7.3×10²² FLOP
Compute estimation method: Hardware
Parameters: 2,100,000,000
Dataset size: 1.4T
Training hardware: Google TPU v4
Chips used: 2,048
Training time: 120 h
Training power draw: 1.4 MW
Training cost (2023 USD): $78K
Model accessibility: Unreleased
Open weights: No
Citations: 1,719
Epoch confidence: Confident