At what point during model development is it best to do model calibration?

Question

I have been working on prediction models in R studio based on a rather small data set. There is a total of ~ 1200 cases with 150 to 400 positive cases depending on which of the different outcomes is being modelled.

Initially it was done with a 70/30 split that was stratified to have a similar distribution of positive cases in both sets, but after gaining knowledge and especially after reading some of Frank Harrel's literature (for instance:"Prediction models need appropriate internal, internal-external, and external validation" by Steyerberg and Harrel) I am considering a different setup.

My main concerns with the work I have done already is with calibration. Brier scores are about 0.10 for the set with a that has ~10% positive classes. As I understand it this implies a non- informative model? AUC approx 0.80, but reliability diagram also implies poor calibration. DCA implies net benefit of using models.

My main question is concerning when to apply calibration to the models under development (GML, XGB, RF). I have seen some setups where people suggest splitting in train - validate - test which is not possible in this case due to data size and would even further decrease the available amount of data for developing models.

1. Can I take the training set of 70% of the data, train models with CV or bootstrapping, take performance metrics from average of the test folds of the cv and finally calibrate the models based on those results before applying them on the hold out set of 30%?

2. Is there another way to perform calibration during model development without having acces to an external validation set?

Otherwise, if you develop a model with poor calibration but an AUC that seem to imply reasonable discrimination, why even continue with external validation if don't know that it could be improved to a well calibrated model that could have real world use..?

Or do you report results including poor calibration and hope that it can be solved at the external validation stage sometime in the future...? That seems like a poor solution!

Thanks in advance for any input/ advice!