Bielik-11B-v2

Parameters

11B

Published

Aug 28, 2024

Bielik-11B-v2 is an AI model developed by SpeakLeash and Cyfronet AGH (Poland), first published in August 2024. It works in the language domain, on tasks such as language modeling/generation and question answering.

Epoch AI has no training-compute estimate for this model. The model has 11,000,000,000 parameters. It was trained on roughly 200B datapoints. Training ran on 256 NVIDIA GH200.

Access: Open weights (unrestricted). Its weights are openly available. It is built on top of Mistral 7B. Epoch AI rates the confidence of this record as confident.

Full record

Organization: SpeakLeash, Cyfronet AGH
Country of organization: Poland
Domain: Language
Task: Language modeling/generation, Question answering
Compute estimation method: Operation counting
Parameters: 11,000,000,000
Dataset size: 200B
Training hardware: NVIDIA GH200
Chips used: 256
Training power draw: 353.2 kW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Base model: Mistral 7B
Epoch confidence: Confident