I have a Multivariate Batch Request problem using the Micrososft Azure Anomaly Detection and though my model was successfuly trained I constantly receive error messages when querying the results. I cannot solve the error myself and thats why I ask for help.
I am new to the Anomaly Detector and I am testing its capabilities in a .NET Application. For comparison and learning purposes I took mainly the example at: https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/quickstarts/client-libraries-multivariate?pivots=programming-language-csharp&tabs=command-line#create-a-new-net-core-application
The modell has been successfully trained, which can be seen here:
10.05.2023 15:02:32 - Multivariate Trainer - Starting the training of a new model
10.05.2023 15:03:15 - Multivariate Trainer - 0 available models before training.
10.05.2023 15:03:15 - Multivariate Trainer - Training new model...(it may take a few minutes)
10.05.2023 15:03:15 - Multivariate Trainer - Training model id is 9c183efc-5578-49e0-9ace-ad77be21f766
10.05.2023 15:04:16 - Multivariate Trainer - try 1, model_id: 9c183efc-5578-49e0-9ace-ad77be21f766, status: Running.
10.05.2023 15:05:16 - Multivariate Trainer - try 2, model_id: 9c183efc-5578-49e0-9ace-ad77be21f766, status: Running.
10.05.2023 15:06:16 - Multivariate Trainer - try 3, model_id: 9c183efc-5578-49e0-9ace-ad77be21f766, status: Running.
10.05.2023 15:07:16 - Multivariate Trainer - try 4, model_id: 9c183efc-5578-49e0-9ace-ad77be21f766, status: Ready.
10.05.2023 15:07:16 - Multivariate Trainer - Creating model succeeds.
10.05.2023 15:07:17 - Multivariate Trainer - 1 available models after training.
The log result of the batch request can be seen here:
10.05.2023 15:09:24 - Start batch detect... 10.05.2023 15:09:24 - Start batch detection, this might take a few minutes...
10.05.2023 15:09:24 - result id is: 8940ca59-724a-4219-96e2-f7b7f007ee0b
10.05.2023 15:09:26 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:27 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:28 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:29 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:30 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:31 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created 10.05.2023 15:09:33 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:34 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:09:35 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Created
10.05.2023 15:10:25 - try: 0, result id: 8940ca59-724a-4219-96e2-f7b7f007ee0b Detection status is Failed
10.05.2023 15:10:25 - Detection failed.10.05.2023 15:10:25 - Errors:
10.05.2023 15:10:25 - Get error message fail: Input string was not in a correct format.
10.05.2023 15:10:25 - Results with ID 8940ca59-724a-4219-96e2-f7b7f007ee0b
The error message leads me to a datasource problem, but I couldn´t identify one.
I was wondering, why the training itself worked though? I have no clue.
I was trying to batch request with different start time and end time values a result for
the whole training data
only for a single day of the training data
only for a time period of 8 hours of the training data
The error always was the same.
Based on the above output I can see, that the error code and message of the resultResponse failed. I don´t know what to do there?
The dataset itself contains
~17k timestamps
of a granularity of 5 seconds
with 6 data series
in a onetable.
All missing timestamps have been filled with zeros and with linear data of the last available value for each data series. No change in the error.
The timestamps have been check for ascending order. I couldn´t find ordering errors.
I also checked for duplicates. I couldn´t find any duplicates.
Here are 20 randomly picked examples with filled "0". Linear fills were tested as well = same error.
2023-05-04T22:02:25Z,47.34,0.00,21.00,0.00,6.00,32.00
2023-05-04T22:02:30Z,44.61,0.00,19.00,0.00,3.00,15.00
2023-05-04T22:02:35Z,39.03,0.00,44.00,0.00,24.00,22.00
2023-05-04T22:02:40Z,38.78,0.00,54.00,0.00,2.00,21.00
2023-05-04T22:02:45Z,36.54,0.00,52.00,0.00,17.00,35.00
2023-05-04T22:02:50Z,39.43,0.00,33.00,0.00,8.00,24.00
2023-05-04T22:02:55Z,35.15,0.00,18.00,0.00,6.00,24.00
2023-05-04T22:03:00Z,37.12,0.00,4.00,0.00,4.00,11.00
2023-05-04T22:03:05Z,30.97,0.00,-42.00,0.00,12.00,14.00
2023-05-04T22:03:10Z,17.18,0.00,-43.00,0.00,10.00,21.00
2023-05-04T22:03:15Z,20.19,0.00,-22.00,0.00,2.00,21.00
2023-05-04T22:03:20Z,19.03,0.00,3.00,0.00,8.00,18.00
2023-05-04T22:03:25Z,23.93,0.00,-5.00,0.00,11.00,19.00
2023-05-04T22:03:30Z,17.54,0.00,-26.00,0.00,21.00,36.00
2023-05-04T22:03:35Z,-4.47,0.00,-24.00,0.00,3.00,32.00
2023-05-04T22:03:40Z,-5.21,0.00,-25.00,0.00,7.00,27.00
2023-05-04T22:03:45Z,-3.91,0.00,7.00,0.00,78.00,33.00
2023-05-04T22:03:50Z,28.32,0.00,1.00,0.00,7.00,34.00
2023-05-04T22:04:00Z,25.68,0.00,2.00,0.00,15.00,11.00
2023-05-04T22:04:05Z,23.20,0.00,-8.00,0.00,16.00,20.00
Your help is highly appreciated. Please don´t hesitate to ask for additional information.
I would expect the Anomaly Detection to deliver me the results I requested without an error message, especially because the training on the exact same datasource/dataset successfully finished.
Do I have some very basic misunderstanding here?