Incoder-6.7B
Incoder-6.7B is an AI model developed by Facebook AI Research, University of Washington, University of California (UC) Berkeley, Carnegie Mellon University (CMU) and Toyota Technological Institute at Chicago (United States and France), first published in April 2023. It works in the language domain, on tasks such as code generation.
Training it took an estimated 3×10²¹ FLOP of compute (estimation method: reported). The model has 6,700,000,000 parameters. It was trained on roughly 52B datapoints. Training ran on NVIDIA V100 for about 576 hours. The compute alone is estimated at $3K in 2023 dollars.
Access: Open weights (non-commercial). Its weights are openly available. The reference paper has 864 citations. Epoch AI rates the confidence of this record as confident.
- Organization
- Facebook AI Research, University of Washington, University of California (UC) Berkeley, Carnegie Mellon University (CMU), Toyota Technological Institute at Chicago
- Country of organization
- United States, France
- Domain
- Language
- Task
- Code generation
- Training compute
- 3×10²¹ FLOP
- Compute estimation method
- Reported
- Parameters
- 6,700,000,000
- Dataset size
- 52B
- Training hardware
- NVIDIA V100
- Training time
- 576 h
- Training cost (2023 USD)
- $3K
- Model accessibility
- Open weights (non-commercial)
- Open weights
- Yes
- Citations
- 864
- Epoch confidence
- Confident