In early 2019, OpenAI proposed GPT-2, a scaled-up version of the GPT-1 model, increasing the number of parameters and the size of the training dataset tenfold.
The number of parameters of this new version was 1.5 billion, trained on 40 GB of text.
In November 2019, OpenAI released the full version of the GPT-2 language model.
GPT-2 is publicly available and can be downloaded from Huggingface or GitHub.
GPT-2 showed that training a larger language model on a larger dataset improves the ability of a language model to understand tasks and outperforms the state-of-art on many jobs.