StableLM-2-1.6B

Training compute

1.9×10²² FLOP

Parameters

1.6B

Published

Jan 18, 2024

StableLM-2-1.6B is an AI model developed by Stability AI (United Kingdom), first published in January 2024. It works in the language domain, on tasks such as language modeling/generation.

Training it took an estimated 1.9×10²² FLOP of compute (estimation method: operation counting). The model has 1,644,417,024 parameters. It was trained on roughly 2T datapoints. Training ran on 512 NVIDIA A100 SXM4 40 GB.

Access: Open weights (non-commercial). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Stability AI
Country of organization: United Kingdom
Domain: Language
Task: Language modeling/generation
Training compute: 1.9×10²² FLOP
Compute estimation method: Operation counting
Parameters: 1,644,417,024
Dataset size: 2T
Training hardware: NVIDIA A100 SXM4 40 GB
Chips used: 512
Training power draw: 405.7 kW
Model accessibility: Open weights (non-commercial)
Open weights: Yes
Epoch confidence: Confident