Orca 2-13B

Training compute

4.6×10²² FLOP

Parameters

13B

Published

Nov 21, 2023

Orca 2-13B is an AI model developed by Microsoft Research (United States), first published in November 2023. It works in the language domain, on tasks such as language modeling/generation and question answering.

Training it took an estimated 4.6×10²² FLOP of compute (estimation method: hardware). The model has 13,000,000,000 parameters. Training ran on NVIDIA A100 SXM4 80 GB for about 80 hours.

Access: Open weights (non-commercial). Its weights are openly available. It is built on top of LLaMA-13B. The reference paper has 202 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Microsoft Research
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Question answering
Training compute: 4.6×10²² FLOP
Compute estimation method: Hardware
Parameters: 13,000,000,000
Training hardware: NVIDIA A100 SXM4 80 GB
Training time: 80 h
Model accessibility: Open weights (non-commercial)
Open weights: Yes
Base model: LLaMA-13B
Citations: 202
Epoch confidence: Confident