Can use different transformer model for tokenizer and model?

Question

Can I use roberta for tokenizer while bert for model?

from transformers import RobertaTokenizerFast

tokenizer = RobertaTokenizerFast.from_pretrained("./bert_tokenizer", max_len=512)

from transformers import BertForMaskedLM
config = BertConfig()
bert= BertForMaskedLM(config)

Answer 1

You cannot use RoBERTa tokenizer for BERT model. The reason is that the vocabulary for BERT and RoBERTa are different. Thus a word-piece token which is present in RoBERTa's vocabulary may not be present in BERT's vocabulary.