Falcon-40B

Training compute

2.4×10²³ FLOP

Parameters

40B

Published

Mar 15, 2023

Falcon-40B is an AI model developed by Technology Innovation Institute (United Arab Emirates), first published in March 2023. It works in the language domain, on tasks such as language modeling, language modeling/generation and question answering.

Training it took an estimated 2.4×10²³ FLOP of compute (estimation method: operation counting,reported). The model has 40,000,000,000 parameters. It was trained on roughly 1T datapoints. Training ran on 384 NVIDIA A100 for about 1.4K hours. The compute alone is estimated at $320K in 2023 dollars.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Technology Innovation Institute
Country of organization: United Arab Emirates
Domain: Language
Task: Language modeling, Language modeling/generation, Question answering
Training compute: 2.4×10²³ FLOP
Compute estimation method: Operation counting, Reported
Parameters: 40,000,000,000
Dataset size: 1T
Training hardware: NVIDIA A100
Chips used: 384
Training time: 1,440 h
Chip-hours: 553K
Training power draw: 306.3 kW
Training cost (2023 USD): $320K
Numerical format: BF16
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 0
Epoch confidence: Confident

Benchmark results

01	Epoch Capabilities Index	ECI Score	104.1
02	HellaSwag	Overall accuracy	0.85
03	BoolQ	Score	0.83
04	PIQA	Score	0.83
05	TriviaQA	EM	0.8
06	LAMBADA	Score	0.77
07	WinoGrande	Accuracy	0.77
08	ARC AI2	Challenge score	0.62
09	MMLU	EM	0.57
10	OpenBookQA	Accuracy	0.57
11	BIG-Bench Hard	Average	0.37
12	GSM8K	EM	0.25

More from Technology Innovation Institute

Falcon-180B Falcon-H1 Falcon3-7B Falcon Mamba Falcon 2 11B Falcon-7B Falcon-Arabic

SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.

← All ai models