LLM Parameter Growth

Phase 1

Explore how language models have grown in size and capability over time

2018

1000B100B10B1B0.1B

0.345B

BERT

20182024

Click on a model bar to see details

Increasing parameters generally increases…

Chinchilla showed that for a fixed compute budget…