Phi-4

Training compute

9.3×10²³ FLOP

Parameters

14B

Published

Dec 12, 2024

Phi-4 is an AI model developed by Microsoft Research (United States), first published in December 2024. It works in the language domain, on tasks such as language modeling/generation, question answering, code generation and quantitative reasoning.

Training it took an estimated 9.3×10²³ FLOP of compute (estimation method: operation counting,hardware). The model has 14,000,000,000 parameters. Training ran on 1,920 NVIDIA H100 SXM5 80GB for about 504 hours.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Microsoft Research
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Question answering, Code generation, Quantitative reasoning
Training compute: 9.3×10²³ FLOP
Compute estimation method: Operation counting, Hardware
Parameters: 14,000,000,000
Training hardware: NVIDIA H100 SXM5 80GB
Chips used: 1,920
Training time: 504 h
Training power draw: 2.6 MW
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Confident

Benchmark results

01	Epoch Capabilities Index	ECI Score	131.1
02	LiveBench	Global average	41.6
03	Lech Mazur Writing	Mean score	6.26
04	MMLU	EM	0.85
05	MATH Level 5	mean_score	0.65
06	GPQA Diamond	mean_score	0.56
07	OTIS Mock AIME 2024-2025	mean_score	0.14
08	Balrog	Average progress	0.12