GQA-8-XXL

Training compute

3.5×10²² FLOP

Parameters

11B

Published

Dec 23, 2023

GQA-8-XXL is an AI model developed by Google Research (United States), first published in December 2023. It works in the language domain, on tasks such as text summarization, language modeling/generation and translation.

Training it took an estimated 3.5×10²² FLOP of compute (estimation method: hardware). The model has 11,000,000,000 parameters. Training ran on Google TPU v3.

Access: Unreleased. Its weights are not openly released. It is built on top of T5-11B. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Google Research
Country of organization: United States
Domain: Language
Task: Text summarization, Language modeling/generation, Translation
Training compute: 3.5×10²² FLOP
Compute estimation method: Hardware
Parameters: 11,000,000,000
Training hardware: Google TPU v3
Chip-hours: 14.4K
Model accessibility: Unreleased
Open weights: No
Base model: T5-11B
Epoch confidence: Confident