AI models
TinyLlama-1.1B (1T token checkpoint)
Training compute
7.2×10²¹ FLOP
Parameters
1.1B
Published
Oct 1, 2023
TinyLlama-1.1B (1T token checkpoint) is an AI model developed by Singapore University of Technology & Design (Singapore), first published in October 2023. It works in the language domain, on tasks such as chat, language modeling/generation, translation and question answering.
Training it took an estimated 7.2×10²¹ FLOP of compute (estimation method: hardware,operation counting). The model has 1,100,000,000 parameters. It was trained on roughly 1T datapoints. Training ran on 16 NVIDIA A100 SXM4 40 GB for about 720 hours.
Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 722 citations. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Singapore University of Technology & Design
- Country of organization
- Singapore
- Domain
- Language
- Task
- Chat, Language modeling/generation, Translation, Question answering
- Training compute
- 7.2×10²¹ FLOP
- Compute estimation method
- Hardware, Operation counting
- Parameters
- 1,100,000,000
- Dataset size
- 1T
- Training hardware
- NVIDIA A100 SXM4 40 GB
- Chips used
- 16
- Training time
- 720 h
- Chip-hours
- 11.5K
- Training power draw
- 12.7 kW
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Citations
- 722
- Epoch confidence
- Confident
More from Singapore University of Technology & Design
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models