NeMO Megatron GPT 20B

Parameters

20B

Published

Sep 15, 2022

NeMO Megatron GPT 20B is an AI model developed by NVIDIA (United States), first published in September 2022. It works in the language domain, on tasks such as language modeling/generation and question answering.

Epoch AI has no training-compute estimate for this model. The model has 20,000,000,000 parameters. It was trained on roughly 341.2B datapoints.

Access: Open weights (unrestricted). Its weights are openly available. Epoch AI rates the confidence of this record as confident.

Full record

Organization: NVIDIA
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Question answering
Parameters: 20,000,000,000
Dataset size: 341.2B
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Epoch confidence: Confident