DELETE index command returns varying JSON objects

Question

(ES 8.6.2, W10)

In Insomnia, when I try to delete a non-existent index, using command DELETE and url https://localhost:9500/my_test_index, I always seem to get a JSON object like this:

{
    "error": {
        "root_cause": [
    ...
    "status": 404
}

Similarly when I do the above in Python, using requests, I get the above response.

But when I run a reqwest request (in Rust) doing that same request I sometimes get the above but often get this:

{
    "acknowledged": true
}

The reason I'm doing this is to delete the index if it exists. Either 200 or 404 would therefore be acceptable status codes. But if I get "acknowledged", true or false, I don't know what that means: it's true even if the index doesn't exist.

This is all happening on a local machine, with one shard. At the current time there are no other indices in my shard. When I create this index, it has a yellow status. So my server/shard seem to be healthy enough.

Can anyone suggest why this might be happening? Might "acknowledged" mean in this case that the server is busy (on a previous command)? Is it possible that Rust's reqwest code works faster than Python?

Edit
I seem to have detected a pattern: I run the caller Python program, which calls the Rust module, and I get this unexplained result, i.e. response.text() is "{"acknowledged": true}". Then I run it again and I get an expected response, with "error" and "status" keys. Then I run again and I get the unexplained result, and so on. I.e. there seems to be an alternation between the 2.

Edit2
If I get this "{ "acknowledged": true }" response.text(), and then repeat the command (whether or not after waiting for 1 ms), it seems that I always get the expected type of response (with "status" key) on the second attempt.

It's funny how this only seems to occur using Rust, not Python. I wonder what the explanation can be. Reading around I see some stuff about "nodes" possibly taking time to respond in the kind of way you expect. Quite strange: would like to know if an expert can suggest what might be going on.

Rust code
As requested. My Rust code takes place inside a utilities function, which uses the duang crate to permit optional/named parameters:

duang!(
    pub fn reqwest_call<T: serde::de::DeserializeOwned + std::fmt::Debug>(url: &str, method: reqwest::Method = reqwest::Method::GET, timeout: u64 = 5_000, 
        acceptable_status_codes: Vec<u16> = vec![200], content_type: &str = "application/json", body_str: &str = "") 
        -> Result<T, Box<dyn std::error::Error>> {
        let mut error_details = format!(
            "=> url was |{url}|\n=> method was {method}\n=> content type was {content_type}\n=> timeout was {timeout}\n=> body\n|");
        if body_str.len() <= 200 {
            error_details.push_str(body_str);
        }
        else {
            error_details.push_str(&(body_str.to_string()[0..200]));
            error_details.push_str("...");
        };
        error_details.push_str("|\n\n");
        let response_result = get_reqwest_client(timeout)?
        .request(method, url)
        .header("Content-type", content_type)
        .body(body_str.to_string())
        .basic_auth("mike12", Some("mike12"))
        .send(); 
        let response = match response_result {
            Ok(response) => response,
            Err(e) => {
                error!("{e}\n{error_details}\n{}", Backtrace::force_capture());
                return Err(Box::new(e))
            }
        };
        ...

... this in turn calls another utility get_reqwest_client which delivers a reqwest.blocking.Client with the specified timeout. The timeout is currently 5 seconds (see default parameter in the above). No timeout error is happening. If it was, send would send us to the Err branch in the above and there'd be no examination of any returned JSON object.

The actual Rust call to my utility method:

    let url = format!("{ES_URL}/{}", &op_handler.index_name);
    #[derive(Deserialize, Debug)]
    struct DeleteResponse {
        status: usize
    }
    let acceptable_status_codes: Vec<u16> = vec![200, 404];
    let _: DeleteResponse = match reqwest_call!(&url, reqwest::Method::DELETE,
        acceptable_status_codes=acceptable_status_codes) {
        Ok(response) => {
            ...
        Err(e) => {
            ... 
            /* this part will reveal the message of error "e": namely 
that the JSON does not have a key "status", which it must have: otherwise
it can't be interpreted (cast) as a `DeleteResponse` `struct`.*/

Answer 1

In most cases you will get with 404 or 200 with "acknowledged": true. You will get "acknowledged": false when index is found and the delete operation is initiated but it takes too long because master node is busy and cannot process the request within the specified timeout. This behavior is not client-specific and there should be no difference between python and rust responses unless you do something different in your request call.

I tried reproducing your situation, unfortunately you omitted some pretty important parts of your code but if you put a print statement after .send(); I get something like this:

Ok(
  Response { 
    url: Url { 
      scheme: "http", 
      cannot_be_a_base: false, 
      username: "", 
      password: None, 
      host: Some(Domain("localhost")),
      port: Some(9500), 
      path: "/test", 
      query: None, 
      fragment: None 
   }, 
   status: 404, 
   headers: {"x-elastic-product": "Elasticsearch", "content-type": "application/json", "content-length": "353"} 
  }
)

And if I put another print statement after match and take a look at response.bytes() you will see something like this:

{
    "error":
    {
        "root_cause":
        [
            {
                "type": "index_not_found_exception",
                "reason": "no such index [test]",
                "resource.type": "index_or_alias",
                "resource.id": "test",
                "index_uuid": "_na_",
                "index": "test"
            }
        ],
        "type": "index_not_found_exception",
        "reason": "no such index [test]",
        "resource.type": "index_or_alias",
        "resource.id": "test",
        "index_uuid": "_na_",
        "index": "test"
    },
    "status": 404
}

So if I replace ... at the end of reqwest with something like

    Ok(serde_json::from_slice(response.bytes().unwrap().as_ref()).unwrap())

I get back Ok(DeleteResponse { status: 404 }) in the caller. It seems like everything is working as expected. What do you get?