Baichuan 2-7B

Training compute

1.1×10²³ FLOP

Parameters

7B

Published

Sep 20, 2023

Baichuan 2-7B is an AI model developed by Baichuan (China), first published in September 2023. It works in the language domain, on tasks such as language modeling/generation, question answering and translation.

Training it took an estimated 1.1×10²³ FLOP of compute (estimation method: operation counting). The model has 7,000,000,000 parameters. It was trained on roughly 2.6T datapoints. Training ran on 1,024 NVIDIA A800 PCIe 40 GB.

Access: Open weights (restricted use). Its weights are openly available. The reference paper has 982 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Baichuan
Country of organization: China
Domain: Language
Task: Language modeling/generation, Question answering, Translation
Training compute: 1.1×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 7,000,000,000
Dataset size: 2.6T
Training hardware: NVIDIA A800 PCIe 40 GB
Chips used: 1,024
Training power draw: 508.4 kW
Model accessibility: Open weights (restricted use)
Open weights: Yes
Citations: 982
Epoch confidence: Confident

Benchmark results

01

Epoch Capabilities Index

ECI Score

96