Composite aggregation

[kəmˈpɑːzət]，组合聚合。属于分桶聚合。

基于不同的源（source）来创建组合聚合（composite aggregation）桶。可以对多级的聚合的结果进行分页。该聚合方式提供了一种方式来流化某种聚合的所有桶，类似于文档的滚动（scroll）。

组合聚合目前不兼容 pipeline aggregation。

组合聚合基于文档的值来创建一个组合，每个组合可以看作是一个组合桶。

比如，文档的内容如下：

{"keyword": ["foo", "bar"],"number": [23, 65, 76]
}

通过使用组合聚合的方式，会产生如下几种组合桶。

{ "keyword": "foo", "number": 23 }
{ "keyword": "foo", "number": 65 }
{ "keyword": "foo", "number": 76 }
{ "keyword": "bar", "number": 23 }
{ "keyword": "bar", "number": 65 }
{ "keyword": "bar", "number": 76 }

• sources：定义聚合源的列表。每个聚合源的名称需要唯一。

• missing_bucket ：默认 false，即如果某个聚合源的结果为空，则整体的组合聚合的结果会输出 []。如果设置 true，只有结果为空的聚合源输出 null，其它聚合源正常输出。

• size：限制组合聚合的结果输出多少条数据。默认 10。

• after：设置当前页的起点，即上一页的最后一条数据。

聚合源

terms、histogram、date_histogram、geotile_grid 四种聚合可以作为聚合源。

terms聚合作为聚合源

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_FlightTimeMin": {"terms": {"field": "FlightTimeMin"}}}
}

这种方式等价于直接使用 terms 聚合。

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"runtime_mappings": {"FlightTimeMinChanged": {"type": "double","script": {"source": """emit(doc['FlightTimeMin'].value / 10)"""}}},"aggs": {"composite_FlightTimeMinChanged": {"composite": {"sources": [{"terms_FlightTimeMinChanged": {"terms": {"field": "FlightTimeMinChanged"}}}]}}}
}

支持运行时字段来创建组合桶。

histogram聚合作为聚合源

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_FlightTimeMin": {"composite": {"sources": [{"histogram_FlightTimeMin": {"histogram": {"field": "FlightTimeMin","interval": 10}}}]}}}
}

date_histogram聚合作为聚合源

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_timestamp": {"composite": {"sources": [{"date_histogram_timestamp": {"date_histogram": {"field": "timestamp","calendar_interval": "1d","format": "yyyy-MM-dd"}}}]}}}
}

多种聚合源组合在一起

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_timestamp_FlightTimeMin": {"composite": {"sources": [{"date_histogram_timestamp": {"date_histogram": {"field": "timestamp","calendar_interval": "1d","format": "yyyy-MM-dd"}}},{"terms_FlightTimeMin": {"terms": {"field": "FlightTimeMin"}}}]}}}
}

不同聚合源分别指定排序规则

先按照第一个聚合源进行排序，然后第二个。。以此类推。

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_timestamp_FlightTimeMin": {"composite": {"sources": [{"date_histogram_timestamp": {"date_histogram": {"field": "timestamp","calendar_interval": "1d","format": "yyyy-MM-dd","order": "desc"}}},{"terms_FlightTimeMin": {"terms": {"field": "FlightTimeMin","order": "asc"}}}]}}}
}

组合聚合与子聚合之间的对比

首先使用组合聚合的方式，按照 OriginCountry、DestCountry 两个字段进行词项聚合。

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_OriginCountry_DestCountry": {"composite": {"sources": [{"terms_OriginCountry": {"terms": {"field": "OriginCountry"}}},{"terms_DestCountry": {"terms": {"field": "DestCountry"}}}]}}}
}

聚合结果如下：

"aggregations" : {"composite_OriginCountry_DestCountry" : {"after_key" : {"terms_OriginCountry" : "AE","terms_DestCountry" : "CA"},"buckets" : [{"key" : {"terms_OriginCountry" : "AE","terms_DestCountry" : "AE"},"doc_count" : 9},{"key" : {"terms_OriginCountry" : "AE","terms_DestCountry" : "AR"},"doc_count" : 10},。。。。。。

作为对比，我们再使用 terms 子聚合的方式。

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"terms_OriginCountry": {"terms": {"field": "OriginCountry"},"aggs": {"terms_DestCountry": {"terms": {"field": "DestCountry"}}}}}
}

聚合结果如下：

"aggregations" : {"terms_OriginCountry" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 4114,"buckets" : [{"key" : "IT","doc_count" : 2278,"terms_DestCountry" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 513,"buckets" : [{"key" : "IT","doc_count" : 459},{"key" : "US","doc_count" : 328},{"key" : "CN","doc_count" : 195},{"key" : "CA","doc_count" : 192},

missing_bucket参数

在第二个聚合源中，我们指定一个不存在的字段 FlightTimeMin2。通过修改 missing_bucket 参数的值，对比它的作用。

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_timestamp_FlightTimeMin": {"composite": {"sources": [{"date_histogram_timestamp": {"date_histogram": {"field": "timestamp","calendar_interval": "1d","format": "yyyy-MM-dd","order": "desc"}}},{"terms_FlightTimeMin": {"terms": {"field": "FlightTimeMin2","order": "asc","missing_bucket": false}}}]}}}
}

after参数

从上一页的 after_key 中，可以得到最后一条数据的内容。

"after_key" : {"date_histogram_timestamp" : "2022-08-28","terms_FlightTimeMin" : 32.9625244140625
}

接下来将 after 参数的内容修改为上述 after_key 的内容，也就是基于上一页来展示下一页的数据内容。

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_timestamp_FlightTimeMin": {"composite": {"size": 5, "after": {"date_histogram_timestamp" : "2022-08-28","terms_FlightTimeMin" : 13.010112762451172}, "sources": [{"date_histogram_timestamp": {"date_histogram": {"field": "timestamp","calendar_interval": "1d","format": "yyyy-MM-dd"}}},{"terms_FlightTimeMin": {"terms": {"field": "FlightTimeMin","missing_bucket": true}}}]}}}
}

支持嵌入子聚合

GET kibana_sample_data_flights/_search
{"track_total_hits": true,"size": 0,"aggs": {"composite_timestamp_FlightTimeMin": {"composite": {"size": 2, "after": {"date_histogram_timestamp" : "2022-08-28","terms_FlightTimeMin" : 13.010112762451172}, "sources": [{"date_histogram_timestamp": {"date_histogram": {"field": "timestamp","calendar_interval": "1d","format": "yyyy-MM-dd"}}},{"terms_FlightTimeMin": {"terms": {"field": "FlightTimeMin","missing_bucket": true}}}]},"aggs": {"stats_FlightTimeMin": {"stats": {"field": "FlightTimeMin"}}}}}
}

聚合结果输出如下：

"aggregations" : {"composite_timestamp_FlightTimeMin" : {"after_key" : {"date_histogram_timestamp" : "2022-08-28","terms_FlightTimeMin" : 17.2014217376709},"buckets" : [{"key" : {"date_histogram_timestamp" : "2022-08-28","terms_FlightTimeMin" : 16.21676254272461},"doc_count" : 1,"stats_FlightTimeMin" : {"count" : 1,"min" : 16.21676254272461,"max" : 16.21676254272461,"avg" : 16.21676254272461,"sum" : 16.21676254272461}},{"key" : {"date_histogram_timestamp" : "2022-08-28","terms_FlightTimeMin" : 17.2014217376709},"doc_count" : 1,"stats_FlightTimeMin" : {"count" : 1,"min" : 17.2014217376709,"max" : 17.2014217376709,"avg" : 17.2014217376709,"sum" : 17.2014217376709}}]}}