RQ-Transformer (1.4B params ImageNet dataset)

Training compute

1.1×10²⁰ FLOP

Parameters

1.4B

Published

Mar 3, 2022

RQ-Transformer (1.4B params ImageNet dataset) is an AI model developed by Kakao and POSTECH (South Korea), first published in March 2022. It works in the vision and image generation domain, on tasks such as text-to-image.

Training it took an estimated 1.1×10²⁰ FLOP of compute (estimation method: hardware). The model has 1,388,000,000 parameters. It was trained on roughly 327.7M datapoints. Training ran on 8 NVIDIA A100 for about 78.44 hours.

The reference paper has 763 citations. Epoch AI rates the confidence of this record as likely.

Full record

Organization: Kakao, POSTECH
Country of organization: South Korea
Domain: Vision, Image generation
Task: Text-to-image
Training compute: 1.1×10²⁰ FLOP
Compute estimation method: Hardware
Parameters: 1,388,000,000
Dataset size: 327.7M
Training hardware: NVIDIA A100
Chips used: 8
Training time: 78 h
Chip-hours: 864
Training power draw: 6.4 kW
Citations: 763
Epoch confidence: Likely