Code Llama-7B

Training compute

1.1×10²³ FLOP

Parameters

7B

Published

Aug 14, 2023

Code Llama-7B is an AI model developed by Meta AI (United States), first published in August 2023. It works in the language domain, on tasks such as code generation.

Training it took an estimated 1.1×10²³ FLOP of compute (estimation method: operation counting). The model has 7,000,000,000 parameters. It was trained on roughly 600B datapoints. Training ran on NVIDIA A100 SXM4 80 GB.

Access: Open weights (restricted use). Its weights are openly available. It is built on top of Llama 2-7B. The reference paper has 3,163 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Meta AI
Country of organization: United States
Domain: Language
Task: Code generation
Training compute: 1.1×10²³ FLOP
Compute estimation method: Operation counting
Parameters: 7,000,000,000
Dataset size: 600B
Training hardware: NVIDIA A100 SXM4 80 GB
Model accessibility: Open weights (restricted use)
Open weights: Yes
Base model: Llama 2-7B
Citations: 3,163
Epoch confidence: Confident