I trained an LSTM model on a 3D dataset and now I’m trying to apply SHAP on the model and the dataset to obtain more advanced insights. I made a lot of tests with all SHAP features but it seems no one support LSTM with 3D datasets. Is there a way to apply SHAP to this kind of model and dataset?
The 3D dataset has the following axis: [instance, timestep, feature] and the LSTM is built to address a multivariate problem.