I am transferring over from caret to the tidymodels environment and I am having an issue with transferring a custom metric. All the available documentation I can find provides examples of building a custom metric using the 'truth' and 'estimate' variables. Accuracy and roc_auc (using truth and estimate) aren't appropriate as the problem involves betting and odds, so the success isn't comparable across events as the odds make one success more favourable than another and I would like to optimise on this.
I have posted to RStudio community with some code (link below) but no success yet, but I guess the question I have is; does anyone know if this is even possible in yardstick currently?
It may be that I simply have to go back to caret.
Thanks Chris