Multi-Token Prediction 7B

Training compute

3.8×10²³ FLOP

Parameters

6.7B

Published

Apr 30, 2024

Multi-Token Prediction 7B is an AI model developed by Facebook AI Research (United States and France), first published in April 2024. It works in the language domain, on tasks such as code generation.

Training it took an estimated 3.8×10²³ FLOP of compute (estimation method: hardware). The model has 6,700,000,000 parameters. Training ran on NVIDIA A100 SXM4 80 GB,NVIDIA H100 SXM5 80GB.

Access: Open weights (non-commercial). Its weights are openly available. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Facebook AI Research
Country of organization: United States, France
Domain: Language
Task: Code generation
Training compute: 3.8×10²³ FLOP
Compute estimation method: Hardware
Parameters: 6,700,000,000
Training hardware: NVIDIA A100 SXM4 80 GB, NVIDIA H100 SXM5 80GB
Model accessibility: Open weights (non-commercial)
Open weights: Yes
Epoch confidence: Likely