nekomata-14b

Training compute

2.6×10²³ FLOP

Parameters

14.2B

Published

Dec 21, 2023

nekomata-14b is an AI model developed by rinna (Japan), first published in December 2023. It works in the language domain, on tasks such as language generation.

Training it took an estimated 2.6×10²³ FLOP of compute (estimation method: operation counting). The model has 14,200,000,000 parameters. It was trained on roughly 66B datapoints. Training ran on 256 Amazon Trainium1 for about 168 hours.

Access: Open weights (restricted use). Its weights are openly available. It is built on top of Qwen-14B. The reference paper has 35 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: rinna
Country of organization: Japan
Domain: Language
Task: Language generation
Training compute: 2.6×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 14,200,000,000
Dataset size: 66B
Training hardware: Amazon Trainium1
Chips used: 256
Training time: 168 h
Numerical format: BF16
Model accessibility: Open weights (restricted use)
Open weights: Yes
Base model: Qwen-14B
Citations: 35
Epoch confidence: Confident