Claude 2 vs GPT-4 (Jun 2023)

Claude 2 (Anthropic) and GPT-4 (Jun 2023) (OpenAI) are both frontier AI models. Claude 2 was published in July 2023 and GPT-4 (Jun 2023) in June 2023.

GPT-4 (Jun 2023) was trained on 2.1×10²⁵ FLOP, about 5.4x the compute of Claude 2 at 3.9×10²⁴ FLOP. Training compute is the closest available proxy for how much was invested in a model, though it says nothing on its own about how well that compute was spent.

GPT-4 (Jun 2023) wins the single benchmark both were scored on. Benchmark counts are a crude scoreboard — the evaluations differ wildly in what they measure and in how saturated they are — so the per-benchmark table below matters more than the tally.