Gemma 2 9B

Training compute

4.3×10²³ FLOP

Parameters

9B

Published

Jun 24, 2024

Gemma 2 9B is an AI model developed by Google DeepMind (United States), first published in June 2024. It works in the language domain, on tasks such as language modeling/generation, chat, code generation and 2 more.

Training it took an estimated 4.3×10²³ FLOP of compute (estimation method: operation counting). The model has 9,000,000,000 parameters. It was trained on roughly 8T datapoints. Training ran on 4,096 Google TPU v4.

Access: Open weights (restricted use). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google DeepMind
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Chat, Code generation, Question answering, Quantitative reasoning
Training compute: 4.3×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 9,000,000,000
Dataset size: 8T
Training hardware: Google TPU v4
Chips used: 4,096
Training power draw: 2.7 MW
Model accessibility: Open weights (restricted use)
Open weights: Yes
Epoch confidence: Confident

Benchmark results

01	Epoch Capabilities Index	ECI Score	119.5
02	BoolQ	Score	0.86
03	GSM8K	EM	0.85
04	PIQA	Score	0.84