AI models
GRITLM 8x7B
Parameters
46.7B
Published
Apr 17, 2024
GRITLM 8x7B is an AI model developed by Contextual AI, The University of Hong Kong and Microsoft (United States and Hong Kong), first published in April 2024. It works in the language domain, on tasks such as language modeling/generation, question answering and quantitative reasoning.
Epoch AI has no training-compute estimate for this model. The model has 46,700,000,000 parameters. Training ran on 8 NVIDIA H100 SXM5 80GB for about 80 hours.
Access: Open weights (unrestricted). Its weights are openly available. It is built on top of Mixtral 8x7B. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Contextual AI, The University of Hong Kong, Microsoft
- Country of organization
- United States, Hong Kong
- Domain
- Language
- Task
- Language modeling/generation, Question answering, Quantitative reasoning
- Compute estimation method
- Hardware
- Parameters
- 46,700,000,000
- Training hardware
- NVIDIA H100 SXM5 80GB
- Chips used
- 8
- Training time
- 80 h
- Chip-hours
- 20.5K
- Training power draw
- 11.1 kW
- Numerical format
- BF16
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Base model
- Mixtral 8x7B
- Epoch confidence
- Confident
More from Contextual AI,The University of Hong Kong,Microsoft
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models