RWKV-5 (Eagle) 7B
RWKV-5 (Eagle) 7B is an AI model developed by RWKV Foundation, EleutherAI, Ohio State University, University of California Santa Barbara (UCSB), Wroclaw Tech (Wrocław University of Science and Technology), Guangdong Laboratory of Artificial Intelligence and Digital Economy (Pazhou Lab) and 9 more (United States, Poland, China, United Kingdom and Canada), first published in September 2024. It works in the language domain, on tasks such as language modeling/generation, question answering, chat and 2 more.
Training it took an estimated 5×10²² FLOP of compute (estimation method: operation counting). The model has 7,520,000,000 parameters. Training ran on 64 NVIDIA H800 SXM5.
Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.
- Organization
- RWKV Foundation, EleutherAI, Ohio State University, University of California Santa Barbara (UCSB), Wroclaw Tech (Wrocław University of Science and Technology), Guangdong Laboratory of Artificial Intelligence and Digital Economy (Pazhou Lab), New York University (NYU), Harvard University, Contextual AI, University of Chinese Academy of Sciences, University of California Santa Cruz, Tsinghua University, University of Edinburgh, University of British Columbia (UBC), Pennsylvania State University
- Country of organization
- United States, Poland, China, United Kingdom, Canada
- Domain
- Language
- Task
- Language modeling/generation, Question answering, Chat, Code generation, Translation
- Training compute
- 5×10²² FLOP
- Compute estimation method
- Operation counting
- Parameters
- 7,520,000,000
- Training hardware
- NVIDIA H800 SXM5
- Chips used
- 64
- Training power draw
- 88.2 kW
- Numerical format
- BF16
- Model accessibility
- Open weights (unrestricted)
- Open weights
- Yes
- Epoch confidence
- Confident