Llama-3-Taiwan-70B

Parameters

70B

Published

Nov 29, 2023

Llama-3-Taiwan-70B is an AI model developed by National Taiwan University (Taiwan), first published in November 2023. It works in the language domain, on tasks such as chat, language modeling/generation and question answering.

Epoch AI has no training-compute estimate for this model. The model has 70,000,000,000 parameters. It was trained on roughly 35.1B datapoints. Training ran on 48 NVIDIA H100 SXM5 80GB.

Access: Open weights (restricted use). Its weights are openly available. It is built on top of Llama 3-70B. Epoch AI rates the confidence of this record as likely.

Full record

Organization: National Taiwan University
Country of organization: Taiwan
Domain: Language
Task: Chat, Language modeling/generation, Question answering
Compute estimation method: Operation counting
Parameters: 70,000,000,000
Dataset size: 35.1B
Training hardware: NVIDIA H100 SXM5 80GB
Chips used: 48
Training power draw: 66.6 kW
Numerical format: BF16
Model accessibility: Open weights (restricted use)
Open weights: Yes
Base model: Llama 3-70B
Epoch confidence: Likely