NVLM-X 72B

Training compute

3×10²⁴ FLOP

Parameters

72B

Published

Oct 22, 2024

NVLM-X 72B is an AI model developed by NVIDIA (United States), first published in October 2024. It works in the vision and language domain, on tasks such as language modeling/generation, vision-language generation, question answering and 3 more.

Training it took an estimated 3×10²⁴ FLOP of compute (estimation method: operation counting). The model has 72,000,000,000 parameters. It was trained on roughly 45.9B datapoints. Training ran on 128 NVIDIA H100 SXM5 80GB.

Access: Unreleased. Its weights are not openly released. It is built on top of Qwen2-72B,InternViT-6B. Epoch AI rates the confidence of this record as likely.

Full record

Organization: NVIDIA
Country of organization: United States
Domain: Vision, Language
Task: Language modeling/generation, Vision-language generation, Question answering, Code generation, Translation, Quantitative reasoning
Training compute: 3×10²⁴ FLOP
Compute estimation method: Operation counting
Parameters: 72,000,000,000
Dataset size: 45.9B
Training hardware: NVIDIA H100 SXM5 80GB
Chips used: 128
Training power draw: 176.4 kW
Numerical format: BF16
Model accessibility: Unreleased
Open weights: No
Base model: Qwen2-72B, InternViT-6B
Epoch confidence: Likely