Elasticsearch query cache not working as expected

Question

I have an index enabled with query cache and it utilizes 10% of the heap memory. To understand the working I have reinitiated the index and cleared all the stats.

This is the stats of cache before executing the query:

"query_cache": {
       "memory_size": "0b",
       "memory_size_in_bytes": 0,
       "total_count": 0,
       "hit_count": 0,
       "miss_count": 0,
       "cache_size": 0,
       "cache_count": 0,
       "evictions": 0
     }

This is the filter context:

{
  "query": {
    "bool": {
      "filter": 
      [{"term": {"id": "1666"}},
      {"terms": {"skip": [0]}}]
      } 
  }
}

This fetches around 4K documents and this is what is the stats after executing it 3 times.

        "query_cache": {
          "memory_size": "6.7kb",
          "memory_size_in_bytes": 6932,
          "total_count": 6,
          "hit_count": 1,
          "miss_count": 5,
          "cache_size": 1,
          "cache_count": 1,
          "evictions": 0
        }

I was expecting the miss_count to be 1 for first time of execution and rest of the times hit_count will be incremented. But it does not seem like it.

Here is the stats of segments

index - my_index 
shard - 0
prirep - p
segment - _jj
generation - 703 
docs.count - 28192
docs.deleted - 190
size - 163.4mb
size.memory - 0
committed - true
searchable - true 
version - 9.5.0
compound - true

Can anyone explain how it works please ?