Yi-34B

Training compute

6.1×10²³ FLOP

Parameters

34B

Published

Nov 2, 2023

Yi-34B is an AI model developed by 01.AI (China), first published in November 2023. It works in the language domain, on tasks such as chat, language modeling/generation, translation and code generation.

Training it took an estimated 6.1×10²³ FLOP of compute (estimation method: operation counting). The model has 34,000,000,000 parameters. It was trained on roughly 3.1T datapoints. Training ran on 128 NVIDIA A100.

Access: Open weights (restricted use). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: 01.AI
Country of organization: China
Domain: Language
Task: Chat, Language modeling/generation, Translation, Code generation
Training compute: 6.1×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 34,000,000,000
Dataset size: 3.1T
Training hardware: NVIDIA A100
Chips used: 128
Training power draw: 101.6 kW
Numerical format: FP16
Model accessibility: Open weights (restricted use)
Open weights: Yes
Epoch confidence: Confident

Benchmark results

01	Epoch Capabilities Index	ECI Score	117.1
02	MMLU	EM	0.76
03	GSM8K	EM	0.67
04	BIG-Bench Hard	Average	0.54