AI models
base LM+GNN+kNN
Training compute
5.3×10¹⁹ FLOP
Parameters
274M
Published
Oct 17, 2021
base LM+GNN+kNN is an AI model developed by Shannon.AI, Nanjing University, Nanyang Technological University and Zhejiang University (ZJU) (China and Singapore), first published in October 2021. It works in the language domain, on tasks such as language modeling.
Training it took an estimated 5.3×10¹⁹ FLOP of compute (estimation method: operation counting). The model has 274,000,000 parameters. It was trained on roughly 103M datapoints.
Access: Open weights (unrestricted). Its weights are openly available. It is built on top of Transformer (Adaptive Input Embeddings) WT103. The reference paper has 46 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Shannon.AI, Nanjing University, Nanyang Technological University, Zhejiang University (ZJU)
- Country of organization
- China, Singapore
- Domain
- Language
- Task
- Language modeling
- Training compute
- 5.3×10¹⁹ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 274,000,000
- Dataset size
- 103M
- Numerical format
- FP16
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Base model
- Transformer (Adaptive Input Embeddings) WT103
- Citations
- 46
- Epoch confidence
- Confident
More from Shannon.AI,Nanjing University,Nanyang Technological University,Zhejiang University (ZJU)
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models