`Hello experts,
I am trying to write BERT model with my own custom model (adding layer end of BERT).It goes well and I would like to save model to avoid future training. I am using trainer API and so, there are two ways to save model, right?
- training_arugments and trainer (save the model under a folder)
This type of saving cannot save config.json although other files can be saved. If I am saving model with only BERT model without adding any layers, it can save all files including json.config.
- trainer.save_model
This also cannot save json.config. So, the question is how can we get all completed saved information for custom model ?? becuase without config.json file we cannot load and resuse the model again. Your suggestions will be highly appreicated how to save the custom model. `
When using the HuggingFace
Trainer
class, the trainer's object typically handle saving the model and its configuration. However, when dealing with custom layers added to a pre-trained model like BERT (such as in your case), you might encounter issues with the automatic saving of theconfig.json
file.To address this, after training, you can manually create a configuration dictionary that includes details about the BERT and your custom added layers, and then use Python's
json
to save the config file. Alternatively, you can create a custom model class that includes your additional layers (which I assume you have already implemented). Then, overridesave_pretrained
method to handle both BERT's configuration and your custom layers.