Designing an algorithm for detecting anamoly and statistical significance for ordinal data using python

Question

Firstly, I would like to apologise for the detailed problem statement. Being a novice, I couldn't express it in any lesser words.

Environment Setup Details:

To give some background, I work in a cloud company where we have multiple servers geographically located in all continents. So, we have hierarchy like this:

1. Several partitions
2. Each partition has 7 pop's
3. Each pop has multiple nodes all set up with redundancy.
4. Turn servers connecting traffic to each node depending on the client location
5. Actual clients-ios, android, mac, windows,etc.

Now, every time the user uses our product/service, he leaves a rating out of 5, 5 being outstanding. This data is stored in our databases and we mine it and analyse it to pin-point the exact issue on any particular day.

For example, if the users from Asia are giving more bad ratings on Tuesday this week than a usual Tuesday, what factors can cause this - is it something to do with clients app version, or server release , physical factors, loss, increased round trip delay etc.

What we have done:

Till now we have been using visualization tools to track each of these metrics separately per day to see the trends and detect the issues manually. But, due to growing micr-services, it is becoming difficult day by day. Now, we want to automate it using python/pandas.

What I want to do:

1. If the ratings drop on a particular day/hour, I run the script and it should do all the manual work by taking all the permutations and combinations of all factors and list out the exact combinations which could have lead to the drop.
2. The second step would be to check whether the drop was significant due to varying number of ratings.

What I know:

I understand that I can do this using pandas by creating a dataframe for each predictor variable and trying to do it per variable. And then I can apply tests like whitney test etc for ordinal data.

What I need help with:

But I just wanted to know if there is a better way to do it? It is perfectly fine if there is a learning curve involved. I can learn and do it. I just wanted some help in choosing the right approach for this.