GPT-2 (AMPS)

Parameters

1.5M

Published

Nov 8, 2021

GPT-2 (AMPS) is an AI model developed by University of California (UC) Berkeley (United States), first published in November 2021. It works in the mathematics and language domain, on tasks such as language modeling/generation and quantitative reasoning.

Epoch AI has no training-compute estimate for this model. The model has 1,500,000 parameters. Training ran on 8 NVIDIA A100 for about 24 hours.

It is built on top of GPT-2 (1.5B). Epoch AI rates the confidence of this record as speculative.

Full record

Organization: University of California (UC) Berkeley
Country of organization: United States
Domain: Mathematics, Language
Task: Language modeling/generation, Quantitative reasoning
Compute estimation method: Hardware
Parameters: 1,500,000
Training hardware: NVIDIA A100
Chips used: 8
Training time: 24 h
Training power draw: 6.5 kW
Base model: GPT-2 (1.5B)
Epoch confidence: Speculative

More from University of California (UC) Berkeley

Denoising Diffusion Probabilistic Models (LSUN Bedroom)Tensor-Transformer(1core)+PN (WT103)Humanoid Locomotion Starling-LM-7B-alpha SqueezeBERT CURL Tensor-Transformer(1core)+PN (PTB)PixelSNAIL (ImageNet)

SourceEpoch AI, 'AI Models'. Published online at epoch.ai. Retrieved 2026-07-29 from https://epoch.ai/data/ai-models. Licensed under CC BY 4.0.

← All ai models