Nemotron-3-8B

Training compute

1.8×10²³ FLOP

Parameters

8B

Published

Nov 15, 2023

Nemotron-3-8B is an AI model developed by NVIDIA (United States), first published in November 2023. It works in the language domain, on tasks such as chat, language generation, language modeling/generation and 3 more.

Training it took an estimated 1.8×10²³ FLOP of compute (estimation method: operation counting,hardware). The model has 8,000,000,000 parameters. It was trained on roughly 3.8T datapoints. Training ran on 1,024 NVIDIA A100 for about 456 hours. The compute alone is estimated at $214K in 2023 dollars.

Access: Open weights (restricted use). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: NVIDIA
Country of organization: United States
Domain: Language
Task: Chat, Language generation, Language modeling/generation, Translation, Code generation, Question answering
Training compute: 1.8×10²³ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 8,000,000,000
Dataset size: 3.8T
Training hardware: NVIDIA A100
Chips used: 1,024
Training time: 456 h
Training power draw: 812.5 kW
Training cost (2023 USD): $214K
Model accessibility: Open weights (restricted use)
Open weights: Yes
Epoch confidence: Confident