The last three years have seen a rapid increase in the size of NLP models

Trends in the size of Natural Language Models by parameters, 2019-2021

Source: Nvidia