Falcon-180B vs Llama 3.1-405B

Falcon-180B (Technology Innovation Institute) and Llama 3.1-405B (Meta AI) are both frontier AI models. Falcon-180B was published in September 2023 and Llama 3.1-405B in July 2024.

Llama 3.1-405B was trained on 3.8×10²⁵ FLOP, about 10.1x the compute of Falcon-180B at 3.8×10²⁴ FLOP. Training compute is the closest available proxy for how much was invested in a model, though it says nothing on its own about how well that compute was spent.

Llama 3.1-405B leads on 6 of the 6 benchmarks they were both scored on, against 0 for Falcon-180B. Benchmark counts are a crude scoreboard — the evaluations differ wildly in what they measure and in how saturated they are — so the per-benchmark table below matters more than the tally.