Mamba2-Hybrid

Training compute

1.8×10²³ FLOP

Parameters

8.7B

Published

Jun 12, 2024

Mamba2-Hybrid is an AI model developed by NVIDIA (United States), first published in June 2024. It works in the language domain, on tasks such as language modeling/generation and question answering.

Training it took an estimated 1.8×10²³ FLOP of compute (estimation method: operation counting). The model has 8,660,000,000 parameters. Training ran on 1,024 NVIDIA H100 SXM5 80GB.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as likely.

Full record

Organization: NVIDIA
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Question answering
Training compute: 1.8×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 8,660,000,000
Training hardware: NVIDIA H100 SXM5 80GB
Chips used: 1,024
Training power draw: 1.4 MW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Likely