Jurassic-1-Jumbo

Training compute

3.7×10²³ FLOP

Parameters

178B

Published

Aug 11, 2021

Jurassic-1-Jumbo is an AI model developed by AI21 Labs (Israel), first published in August 2021. It works in the language domain, on tasks such as language modeling/generation and chat. It counts among the frontier models: the systems trained with the most compute of their moment.

Training it took an estimated 3.7×10²³ FLOP of compute (estimation method: third-party estimation). The model has 178,000,000,000 parameters. It was trained on roughly 300B datapoints. Training ran on NVIDIA A100. The compute alone is estimated at $837K in 2023 dollars.

Access: API access. Its weights are not openly released. The reference paper has 55 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: AI21 Labs
Country of organization: Israel
Domain: Language
Task: Language modeling/generation, Chat
Training compute: 3.7×10²³ FLOP
Compute estimation method: Third-party estimation
Parameters: 178,000,000,000
Dataset size: 300B
Training hardware: NVIDIA A100
Training cost (2023 USD): $837K
Model accessibility: API access
Open weights: No
Citations: 55
Epoch confidence: Confident