DeepSeek-VL-7B

Training compute

10²³ FLOP

Parameters

7B

Published

Mar 8, 2024

DeepSeek-VL-7B is an AI model developed by DeepSeek (China), first published in March 2024. It works in the multimodal, vision and language domain, on tasks such as character recognition (ocr), language modeling/generation, visual question answering and question answering.

Training it took an estimated 10²³ FLOP of compute (estimation method: operation counting,hardware). The model has 7,000,000,000 parameters. It was trained on roughly 400M datapoints. Training ran on 512 NVIDIA A100 for about 120 hours.

Access: Open weights (restricted use). Its weights are openly available. It is built on top of DeepSeek LLM 7B. The reference paper has 797 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: DeepSeek
Country of organization: China
Domain: Multimodal, Vision, Language
Task: Character recognition (OCR), Language modeling/generation, Visual question answering, Question answering
Training compute: 10²³ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 7,000,000,000
Dataset size: 400M
Training hardware: NVIDIA A100
Chips used: 512
Training time: 120 h
Training power draw: 405.2 kW
Model accessibility: Open weights (restricted use)
Open weights: Yes
Base model: DeepSeek LLM 7B
Citations: 797
Epoch confidence: Confident