AI models
HGRN 1B (WT 103)
Training compute
2×10¹⁹ FLOP
Parameters
1B
Published
Nov 8, 2023
HGRN 1B (WT 103) is an AI model developed by Shanghai AI Lab and Massachusetts Institute of Technology (MIT) (China and United States), first published in November 2023. It works in the language domain, on tasks such as language modeling/generation and question answering.
Training it took an estimated 2×10¹⁹ FLOP of compute (estimation method: operation counting). The model has 1,000,000,000 parameters. It was trained on roughly 100B datapoints. Training ran on 8 NVIDIA A100.
Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Shanghai AI Lab, Massachusetts Institute of Technology (MIT)
- Country of organization
- China, United States
- Domain
- Language
- Task
- Language modeling/generation, Question answering
- Training compute
- 2×10¹⁹ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 1,000,000,000
- Dataset size
- 100B
- Training hardware
- NVIDIA A100
- Chips used
- 8
- Training power draw
- 6.3 kW
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models