Kosmos-2

Training compute

4.6×10²⁰ FLOP

Parameters

1.6B

Published

Jun 26, 2023

Kosmos-2 is an AI model developed by Microsoft (United States), first published in June 2023. It works in the language, vision and multimodal domain, on tasks such as visual question answering, image captioning, named entity recognition (ner) and 2 more.

Training it took an estimated 4.6×10²⁰ FLOP of compute (estimation method: operation counting,hardware). The model has 1,600,000,000 parameters. It was trained on roughly 25B datapoints. Training ran on 256 NVIDIA V100 for about 24 hours.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 1,180 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Microsoft
Country of organization: United States
Domain: Language, Vision, Multimodal
Task: Visual question answering, Image captioning, Named entity recognition (NER), Character recognition (OCR), Document representation
Training compute: 4.6×10²⁰ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 1,600,000,000
Dataset size: 25B
Training hardware: NVIDIA V100
Chips used: 256
Training time: 24 h
Chip-hours: 6.1K
Training power draw: 152.8 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 1,180
Epoch confidence: Likely