elastic search - using a nested filtered array as bucket

Question

I'm a bit lost ...

Consider this simple indexed document :

{
"url" : "http://...?mypage"
"pages": [
 {
  "elapsed": 1190,
  "type": "LOADPAGE"
 },
 {
  "elapsed": 115400,
  "type": "ONPAGE"
 },
 {
  "elapsed": 1100,
  "type": "LOADPAGE"
 },
 {
  "elapsed": 1340,
  "type": "ONPAGE"
 }
]    
}

I'm trying to compute the average LOADPAGE, so I know that I will need the "avg" or "stats" aggregation.

"aggs": {
    "compute_loadpage": {
        "filter": { "term": { "pages.type": "loadpage" } },
        "aggs": {
            "loadpage_all": {
                "stats": {
                    "field": "pages.elapsed"
                }
            }
       }
    }
}

I know that the "filter" agg will create a bucket with all documents corresponding to my filter, then it is understandable that my agg will be done on my full "pages" array.

How can I create a bucket with only LOADPAGE values, then I will be able to agg on it , or must I use a scripted agg ?

Answer 1

You can do it with a nested aggregation as long as your document mapping uses a nested type.

To test, I set up a simple index like this (note the nested type, and "index": "not_analyzed" on "pages.type"):

PUT /test_index
{
   "settings": {
      "number_of_shards": 1
   },
   "mappings": {
      "doc": {
         "properties": {
            "pages": {
               "type": "nested",
               "properties": {
                  "elapsed": {
                     "type": "long"
                  },
                  "type": {
                     "type": "string",
                     "index": "not_analyzed"
                  }
               }
            },
            "url": {
               "type": "string"
            }
         }
      }
   }
}

Then I indexed your document:

PUT /test_index/doc/1
{
   "url": "http://...?mypage",
   "pages": [
      {
         "elapsed": 1190,
         "type": "LOADPAGE"
      },
      {
         "elapsed": 115400,
         "type": "ONPAGE"
      },
      {
         "elapsed": 1100,
         "type": "LOADPAGE"
      },
      {
         "elapsed": 1340,
         "type": "ONPAGE"
      }
   ]
}

Then this aggregation seems to provide what you are wanting:

POST /test_index/_search?search_type=count
{
   "aggs": {
      "pages_nested": {
         "nested": {
            "path": "pages"
         },
         "aggs": {
            "loadpage_filtered": {
               "filter": {
                  "term": {
                     "pages.type": "LOADPAGE"
                  }
               },
               "aggs": {
                  "loadpage_avg_elap": {
                     "avg": {
                        "field": "pages.elapsed"
                     }
                  }
               }
            }
         }
      }
   }
}
...
{
   "took": 3,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 1,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "pages_nested": {
         "doc_count": 4,
         "loadpage_filtered": {
            "doc_count": 2,
            "loadpage_avg_elap": {
               "value": 1145,
               "value_as_string": "1145.0"
            }
         }
      }
   }
}

Here is the code I used to test:

http://sense.qbox.io/gist/b526427f14225b02e7268ed15d8c6dde4793fc8d