results highlight the importance of previously overlooked design choices, and raise questions about the source Nevertheless, in the vocabulary size growth in RoBERTa allows to encode almost any word or subword without using the unknown token, compared to BERT. This gives a consider