Gemma 2 27B

Training compute

2.1×10²⁴ FLOP

Parameters

27B

Published

Jun 24, 2024

Gemma 2 27B is an AI model developed by Google DeepMind (United States), first published in June 2024. It works in the language domain, on tasks such as language modeling/generation, chat, code generation and 2 more.

Training it took an estimated 2.1×10²⁴ FLOP of compute (estimation method: operation counting). The model has 27,000,000,000 parameters. It was trained on roughly 13T datapoints. Training ran on 6,144 Google TPU v5p.

Access: Open weights (restricted use). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google DeepMind
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Chat, Code generation, Question answering, Quantitative reasoning
Training compute: 2.1×10²⁴ FLOP
Compute estimation method: Operation counting
Parameters: 27,000,000,000
Dataset size: 13T
Training hardware: Google TPU v5p
Chips used: 6,144
Training power draw: 6.5 MW
Model accessibility: Open weights (restricted use)
Open weights: Yes
Epoch confidence: Confident

Benchmark results

01

Epoch Capabilities Index

ECI Score

122.5