DNABERT

Training compute

1.1×10²⁰ FLOP

Parameters

110M

Published

Aug 15, 2021

DNABERT is an AI model developed by Northeastern University (United States), first published in August 2021. It works in the biology domain, on tasks such as protein or nucleotide language model (plm/nlm).

Training it took an estimated 1.1×10²⁰ FLOP of compute (estimation method: hardware,operation counting). The model has 110,000,000 parameters. It was trained on roughly 1.4B datapoints. Training ran on NVIDIA GeForce RTX 2080 Ti 11GB for about 600 hours.

Access: Open weights (unrestricted). Its weights are openly available. The reference paper has 958 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Northeastern University
Country of organization: United States
Domain: Biology
Task: Protein or nucleotide language model (pLM/nLM)
Training compute: 1.1×10²⁰ FLOP
Compute estimation method: Hardware, Operation counting
Parameters: 110,000,000
Dataset size: 1.4B
Training hardware: NVIDIA GeForce RTX 2080 Ti 11GB
Training time: 600 h
Numerical format: FP16
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Citations: 958
Epoch confidence: Confident