400 BAD Request attempting to contact Azure Anomaly Detection API

Question

I have been following the tutorial listed here - https://learn.microsoft.com/en-us/azure/cognitive-services/anomaly-detector/quickstarts/detect-data-anomalies-csharp

I have set up my Anomaly API so I have the end points and API keys ready to go. I have set up my own time series data which I will post here for clarity (Note: While the below does NOT work, the sample data here using the same code DOES work - https://github.com/Azure-Samples/anomalydetector/blob/master/example-data/request-data.json):

{
  "granularity": "daily",
  "series": [
    {
      "timestamp": "2019-03-27T00:00:00",
      "value": 10781.70
    },
    {
      "timestamp": "2019-03-25T00:00:00",
      "value": 4058.13
    },
    {
      "timestamp": "2019-03-20T00:00:00",
      "value": 8124132.33
    },
    {
      "timestamp": "2019-03-19T00:00:00",
      "value": 1571398.97
    },
    {
      "timestamp": "2019-03-18T00:00:00",
      "value": 2097703.40
    },
    {
      "timestamp": "2019-03-15T00:00:00",
      "value": 10624.76
    },
    {
      "timestamp": "2019-03-14T00:00:00",
      "value": 11647.00
    },
    {
      "timestamp": "2019-03-13T00:00:00",
      "value": 45937.16
    },
    {
      "timestamp": "2019-03-08T00:00:00",
      "value": 4237.20
    },
    {
      "timestamp": "2019-03-07T00:00:00",
      "value": 3315.40
    },
    {
      "timestamp": "2019-03-04T00:00:00",
      "value": 3218.77
    },
    {
      "timestamp": "2019-02-28T00:00:00",
      "value": 11271.00
    },
    {
      "timestamp": "2019-02-27T00:00:00",
      "value": 48605.08
    },
    {
      "timestamp": "2019-02-26T00:00:00",
      "value": 6181.12
    },
    {
      "timestamp": "2019-02-25T00:00:00",
      "value": 45069.00
    },
    {
      "timestamp": "2019-02-22T00:00:00",
      "value": 108860.84
    },
    {
      "timestamp": "2019-02-21T00:00:00",
      "value": 24924.50
    },
    {
      "timestamp": "2019-02-20T00:00:00",
      "value": 4068.50
    },
    {
      "timestamp": "2019-02-19T00:00:00",
      "value": 4329.60
    },
    {
      "timestamp": "2019-02-18T00:00:00",
      "value": 7615.20
    },
    {
      "timestamp": "2019-02-14T00:00:00",
      "value": 56974.10
    },
    {
      "timestamp": "2019-02-13T00:00:00",
      "value": 73393.52
    },
    {
      "timestamp": "2019-02-12T00:00:00",
      "value": 29991.99
    },
    {
      "timestamp": "2019-02-11T00:00:00",
      "value": 2906769.50
    },
    {
      "timestamp": "2019-02-05T00:00:00",
      "value": 1956853.85
    },
    {
      "timestamp": "2019-02-04T00:00:00",
      "value": 46863.31
    },
    {
      "timestamp": "2019-01-31T00:00:00",
      "value": 31602.31
    },
    {
      "timestamp": "2019-01-30T00:00:00",
      "value": 13149.59
    },
    {
      "timestamp": "2018-10-10T00:00:00",
      "value": 19380.60
    },
    {
      "timestamp": "2018-08-21T00:00:00",
      "value": 61801.45
    },
    {
      "timestamp": "2018-08-16T00:00:00",
      "value": 843.80
    },
    {
      "timestamp": "2018-08-15T00:00:00",
      "value": 52326.20
    },
    {
      "timestamp": "2018-08-14T00:00:00",
      "value": 136384.88
    },
    {
      "timestamp": "2018-08-09T00:00:00",
      "value": 7224.30
    },
    {
      "timestamp": "2018-07-26T00:00:00",
      "value": 16493.08
    },
    {
      "timestamp": "2018-07-24T00:00:00",
      "value": 1665163.72
    },
    {
      "timestamp": "2018-07-23T00:00:00",
      "value": 38642.88
    },
    {
      "timestamp": "2018-07-13T00:00:00",
      "value": 49913.00
    },
    {
      "tim estamp": "2018-07-12T00:00:00",
      "value": 49193.00
    },
    {
      "timestamp": "2018-07-11T00:00:00",
      "value": 37205.30
    },
    {
      "timestamp": "2018-07-10T00:00:00",
      "value": 44527.30
    },
    {
      "timestamp": "2018-07-09T00:00:00",
      "value": 148737.01
    },
    {
      "timestamp": "2018-07-06T00:00:00",
      "value": 138887.90
    },
    {
      "timestamp": "2018-07-05T00:00:00",
      "value": 74346.00
    },
    {
      "timestamp": "2018-07-04T00:00:00",
      "value": 71181.50
    },
    {
      "timestamp": "2018-07-03T00:00:00",
      "value": 215164.43
    },
    {
      "timestamp": "2018-07-02T00:00:00",
      "value": 83817.50
    }
  ]
}

When I run my test code, I get back a 400 BAD REQUEST (with no additional information to suggest why). Here is my console app code (with obvious parts redacted)

using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;

namespace anomalyDetectionMachineLearning
{
    class Program
    {
        // Replace the subscriptionKey string value with your valid subscription key.
        const string subscriptionKey = "";
        // Replace the endpoint URL with the correct one for your subscription. 
        // Your endpoint can be found in the Azure portal. For example: https://westus2.api.cognitive.microsoft.com
        const string endpoint = "https://westus2.api.cognitive.microsoft.com/";
        // Replace the dataPath string with a path to the JSON formatted time series data.
        const string dataPath = @"C:\Temp\data.txt";
        const string latestPointDetectionUrl = "/anomalydetector/v1.0/timeseries/last/detect"; 
        const string batchDetectionUrl = "/anomalydetector/v1.0/timeseries/entire/detect";

        static void Main(string[] args)
        {

            try
            {
                var requestData = File.ReadAllText(dataPath);
                //Console.Write(requestData.ToString());
                detectAnomaliesBatch(requestData);
                //detectAnomaliesLatest(requestData);

                System.Console.ReadKey();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
                Console.ReadKey(); 
            }
        }

        public static void detectAnomaliesBatch(string requestData)
        {
            System.Console.WriteLine("Detecting anomalies as a batch");

            var result = Request(
                endpoint,
                batchDetectionUrl,
                subscriptionKey,
                requestData).Result;

            dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(result);
            System.Console.WriteLine(jsonObj);

            bool[] anomalies = jsonObj["isAnomaly"].ToObject<bool[]>();
            System.Console.WriteLine("\n Anomalies detected in the following data positions:");
            for (var i = 0; i < anomalies.Length; i++)
            {
                if (anomalies[i])
                {
                    System.Console.Write(i + ", ");
                }
            }
        }

        static async Task<string> Request(string baseAddress, string endpoint, string subscriptionKey, string requestData)
        {
            using (HttpClient client = new HttpClient { BaseAddress = new Uri(baseAddress) })
            {
                System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);

                var content = new StringContent(requestData, Encoding.UTF8, "application/json");
                var res = await client.PostAsync(endpoint, content);
                if (res.IsSuccessStatusCode)
                {
                    return await res.Content.ReadAsStringAsync();
                }
                else
                {
                    Console.WriteLine($"{res.Content.ToString()}");
                    Console.ReadKey();
                    return $"ErrorCode: {res.StatusCode}";
                }
            }
        }
    }
}

Here is the working sample data that there are no issues using (pasted here at request of someone in comments)

{
    "granularity": "daily",
    "series": [
      {
        "timestamp": "2018-03-01T00:00:00Z",
        "value": 32858923
      },
      {
        "timestamp": "2018-03-02T00:00:00Z",
        "value": 29615278
      },
      {
        "timestamp": "2018-03-03T00:00:00Z",
        "value": 22839355
      },
      {
        "timestamp": "2018-03-04T00:00:00Z",
        "value": 25948736
      },
      {
        "timestamp": "2018-03-05T00:00:00Z",
        "value": 34139159
      },
      {
        "timestamp": "2018-03-06T00:00:00Z",
        "value": 33843985
      },
      {
        "timestamp": "2018-03-07T00:00:00Z",
        "value": 33637661
      },
      {
        "timestamp": "2018-03-08T00:00:00Z",
        "value": 32627350
      },
      {
        "timestamp": "2018-03-09T00:00:00Z",
        "value": 29881076
      },
      {
        "timestamp": "2018-03-10T00:00:00Z",
        "value": 22681575
      },
      {
        "timestamp": "2018-03-11T00:00:00Z",
        "value": 24629393
      },
      {
        "timestamp": "2018-03-12T00:00:00Z",
        "value": 34010679
      },
      {
        "timestamp": "2018-03-13T00:00:00Z",
        "value": 33893888
      },
      {
        "timestamp": "2018-03-14T00:00:00Z",
        "value": 33760076
      },
      {
        "timestamp": "2018-03-15T00:00:00Z",
        "value": 33093515
      },
      {
        "timestamp": "2018-03-16T00:00:00Z",
        "value": 29945555
      },
      {
        "timestamp": "2018-03-17T00:00:00Z",
        "value": 22676212
      },
      {
        "timestamp": "2018-03-18T00:00:00Z",
        "value": 25262514
      },
      {
        "timestamp": "2018-03-19T00:00:00Z",
        "value": 33631649
      },
      {
        "timestamp": "2018-03-20T00:00:00Z",
        "value": 34468310
      },
      {
        "timestamp": "2018-03-21T00:00:00Z",
        "value": 34212281
      },
      {
        "timestamp": "2018-03-22T00:00:00Z",
        "value": 38144434
      },
      {
        "timestamp": "2018-03-23T00:00:00Z",
        "value": 34662949
      },
      {
        "timestamp": "2018-03-24T00:00:00Z",
        "value": 24623684
      },
      {
        "timestamp": "2018-03-25T00:00:00Z",
        "value": 26530491
      },
      {
        "timestamp": "2018-03-26T00:00:00Z",
        "value": 35445003
      },
      {
        "timestamp": "2018-03-27T00:00:00Z",
        "value": 34250789
      },
      {
        "timestamp": "2018-03-28T00:00:00Z",
        "value": 33423012
      },
      {
        "timestamp": "2018-03-29T00:00:00Z",
        "value": 30744783
      },
      {
        "timestamp": "2018-03-30T00:00:00Z",
        "value": 25825128
      },
      {
        "timestamp": "2018-03-31T00:00:00Z",
        "value": 21244209
      },
      {
        "timestamp": "2018-04-01T00:00:00Z",
        "value": 22576956
      },
      {
        "timestamp": "2018-04-02T00:00:00Z",
        "value": 31957221
      },
      {
        "timestamp": "2018-04-03T00:00:00Z",
        "value": 33841228
      },
      {
        "timestamp": "2018-04-04T00:00:00Z",
        "value": 33554483
      },
      {
        "timestamp": "2018-04-05T00:00:00Z",
        "value": 32383350
      },
      {
        "timestamp": "2018-04-06T00:00:00Z",
        "value": 29494850
      },
      {
        "timestamp": "2018-04-07T00:00:00Z",
        "value": 22815534
      },
      {
        "timestamp": "2018-04-08T00:00:00Z",
        "value": 25557267
      },
      {
        "timestamp": "2018-04-09T00:00:00Z",
        "value": 34858252
      },
      {
        "timestamp": "2018-04-10T00:00:00Z",
        "value": 34750597
      },
      {
        "timestamp": "2018-04-11T00:00:00Z",
        "value": 34717956
      },
      {
        "timestamp": "2018-04-12T00:00:00Z",
        "value": 34132534
      },
      {
        "timestamp": "2018-04-13T00:00:00Z",
        "value": 30762236
      },
      {
        "timestamp": "2018-04-14T00:00:00Z",
        "value": 22504059
      },
      {
        "timestamp": "2018-04-15T00:00:00Z",
        "value": 26149060
      },
      {
        "timestamp": "2018-04-16T00:00:00Z",
        "value": 35250105
      }
    ]
  }

Answer 1

There some rules you should strictly follow -

1. File should contain two and only two columns. timestamp and value all in small letters
2. timestamp should be in ISO 8601 format.

Answer 2

According to the 400 Possible Errors in this doc https://westus2.dev.cognitive.microsoft.com/docs/services/AnomalyDetector/operations/post-timeseries-entire-detect, it seems that there are two issues in your data.

1. The timestamps are not in ascending order
2. There seems to be more than 10% points missing in the given time range.

Answer 3

I investigated the sample, the reason you could not see error message because code here should be

 if (res.IsSuccessStatusCode)
 {
    return await res.Content.ReadAsStringAsync();
 }
 else
 {
     Console.WriteLine(res.StatusCode);
     return await res.Content.ReadAsStringAsync();
 }

The reason why the first request failed because this

{
  "tim estamp": "2018-07-12T00:00:00",
  "value": 49193.00
}

But it should be

{
  "timestamp": "2018-07-12T00:00:00",
  "value": 49193.00
}

However, the request still could not be submitted successfully, as anomaly detector service require the input request sorted by timestamp in ascending order and the missing value rate of the input time-series can not exceed 10%. In API reference and best practice part you could find more useful information about the service.

Thanks for your feedback, as we also released .Net SDK. You may consider using the SDK to handle the client work. we will also update quick start part to cover the failed requests error message.