NVIDIA A100 SXM4 40 GB

FP16 throughput

3.1×10¹⁴ FLOP/s

Memory

37.3 GB

TDP

400 W

NVIDIA A100 SXM4 40 GB is an AI accelerator designed by NVIDIA (GPU), released in May 2020. It is built on a 7 nm process at TSMC, packing 54.2B transistors. Its release price was $15K.

It delivers 3.1×10¹⁴ FLOP/s of tensor FP16/BF16 throughput. It carries 37.3 GB of memory at 1.4 TB/s of bandwidth. Its thermal design power is 400 W.

Full record

Manufacturer: NVIDIA
Type: GPU
Release date: May 14, 2020
Release price: $15K
Tensor FP16/BF16: 3.1×10¹⁴ FLOP/s
FP32: 1.9×10¹³ FLOP/s
FP64: 9.7×10¹² FLOP/s
INT8: 6.2×10¹⁴ OP/s
Memory: 37.3 GB
Memory bandwidth: 1.4 TB
TDP: 400 W
Process node: 7 nm
Transistors (millions): 54,200
Die size (mm2): 826
Foundry: TSMC
Tensor cores: 432