Flan T5-XXL + BLIP-2

Parameters

12.1B

Published

Jan 30, 2023

Flan T5-XXL + BLIP-2 is an AI model developed by Salesforce Research (United States), first published in January 2023. It works in the multimodal, language and vision domain, on tasks such as vision-language generation, chat and visual question answering.

Epoch AI has no training-compute estimate for this model. The model has 12,100,000,000 parameters. Training ran on NVIDIA A100 SXM4 40 GB for about 200 hours. The compute alone is estimated at $100K in 2023 dollars.

Access: Open weights (unrestricted). Its weights are openly available. It is built on top of Flan-T5 11B. The reference paper has 8,019 citations. Epoch AI rates the confidence of this record as confident.

Full record

Organization: Salesforce Research
Country of organization: United States
Domain: Multimodal, Language, Vision
Task: Vision-language generation, Chat, Visual question answering
Parameters: 12,100,000,000
Training hardware: NVIDIA A100 SXM4 40 GB
Training time: 200 h
Training cost (2023 USD): $100K
Numerical format: BF16
Model accessibility: Open weights (unrestricted)
Open weights: Yes
Base model: Flan-T5 11B
Citations: 8,019
Epoch confidence: Confident