AI models
BitNet b1.58
Training compute
2.9×10²² FLOP
Parameters
70B
Published
Feb 27, 2024
BitNet b1.58 is an AI model developed by University of Chinese Academy of Sciences and Microsoft Research (China and United States), first published in February 2024. It works in the language domain, on tasks such as language modeling/generation and question answering.
Training it took an estimated 2.9×10²² FLOP of compute (estimation method: operation counting,comparison with other models). The model has 70,000,000,000 parameters. It was trained on roughly 100B datapoints.
Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- University of Chinese Academy of Sciences, Microsoft Research
- Country of organization
- China, United States
- Domain
- Language
- Task
- Language modeling/generation, Question answering
- Training compute
- 2.9×10²² FLOP
- Compute estimation method
- Operation counting, Comparison with other models
- Parameters
- 70,000,000,000
- Dataset size
- 100B
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models