AI models
RoFormer
Training compute
2.2×10¹⁸ FLOP
Parameters
110M
Published
Nov 8, 2023
RoFormer is an AI model developed by Zhuiyi Technology (China), first published in November 2023. It works in the language domain, on tasks such as text classification and language modeling/generation.
Training it took an estimated 2.2×10¹⁸ FLOP of compute (estimation method: operation counting). The model has 110,000,000 parameters. It was trained on roughly 3.3B datapoints. Training ran on 4 NVIDIA V100.
Access: Unreleased. Its weights are not openly released. Epoch AI rates the confidence of this record as confident.
Full record
- Organization
- Zhuiyi Technology
- Country of organization
- China
- Domain
- Language
- Task
- Text classification, Language modeling/generation
- Training compute
- 2.2×10¹⁸ FLOP
- Compute estimation method
- Operation counting
- Parameters
- 110,000,000
- Dataset size
- 3.3B
- Training hardware
- NVIDIA V100
- Chips used
- 4
- Training power draw
- 2.4 kW
- Model accessibility
- Unreleased
- Open weights
- No
- Epoch confidence
- Confident
SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.
← All ai models