Hermes 3 405B

Parameters

405B

Published

Aug 14, 2024

Hermes 3 405B is an AI model developed by Nous Research (United States), first published in August 2024. It works in the language domain, on tasks such as language modeling/generation, chat and question answering.

Epoch AI has no training-compute estimate for this model. The model has 405,000,000,000 parameters. It was trained on roughly 270M datapoints. Training ran on 128 NVIDIA H100 SXM5 80GB.

Access: Open weights (restricted use). Its weights are openly available. It is built on top of Llama 3.1-405B. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Nous Research
Country of organization: United States
Domain: Language
Task: Language modeling/generation, Chat, Question answering
Compute estimation method: Hardware, Operation counting
Parameters: 405,000,000,000
Dataset size: 270M
Training hardware: NVIDIA H100 SXM5 80GB
Chips used: 128
Chip-hours: 2.1K
Training power draw: 176.7 kW
Model accessibility: Open weights (restricted use)
Open weights: Yes
Base model: Llama 3.1-405B
Epoch confidence: Confident