I've been working on a random forest classifier model in bqml which is supposed to detect risky credit card transactions, and one of the features is 'payment_method', which indicates the type of the CC.
When I tested it on "real world" transactions, I noticed that it gave an extremely high score to transactions with a new category of 'payment_method', one that the model hasn't been trained on, but still shouldn't be risky as the model predicted.
Any ideas why this is happening and how I can make sure that next times it encounters new categories, it won't trigger a high score?
(Btw, the automatic encoding in bqml in random forest is label, if it matters somehow.)
The bqml model shouldn't have predicted such a high score to a category that it hasn't been trained on, but instead, understand that this category is legit.