BERT from scratch Training a BERT model from scratch This example is based on a part of the OSCAR dataset (https://oscar-corpus.com/) and the Hugging Face transformers library.