NVILA 8B
NVILA 8B is an AI model developed by NVIDIA, Massachusetts Institute of Technology (MIT), University of California (UC) Berkeley, University of California San Diego, University of Washington and Tsinghua University (United States and China), first published in December 2024. It works in the vision, language, multimodal and video domain, on tasks such as visual question answering and video description.
Training it took an estimated 2.3×10²¹ FLOP of compute (estimation method: hardware). The model has 8,000,000,000 parameters. It was trained on roughly 47.5B datapoints. Training ran on 128 NVIDIA H100 SXM5 80GB for about 16.7 hours.
Access: Open weights (non-commercial). Its weights are openly available. Epoch AI rates the confidence of this record as likely.
- Organization
- NVIDIA, Massachusetts Institute of Technology (MIT), University of California (UC) Berkeley, University of California San Diego, University of Washington, Tsinghua University
- Country of organization
- United States, China
- Domain
- Vision, Language, Multimodal, Video
- Task
- Visual question answering, Video description
- Training compute
- 2.3×10²¹ FLOP
- Compute estimation method
- Hardware
- Parameters
- 8,000,000,000
- Dataset size
- 47.5B
- Training hardware
- NVIDIA H100 SXM5 80GB
- Chips used
- 128
- Training time
- 17 h
- Chip-hours
- 2.1K
- Training power draw
- 176.2 kW
- Numerical format
- FP8
- Model accessibility
- Open weights (non-commercial)
- Open weights
- Yes
- Epoch confidence
- Likely
| 01 | Video-MME | Overall (no subtitles) | 0.64 |