当前位置:网站首页>Elasticsearch Part 6: aggregate statistical query
Elasticsearch Part 6: aggregate statistical query
2020-11-06 20:10:00 【itread01】
There has been no record before Elasticsearch Aggregate queries or other complex queries . Take notes in this article , To facilitate testing , The index data is still the test index library generated in Article 5 db_student_test , The nickname is student_test
The first part Basic aggregation
1、 Maximum max、 minimum value min、 Average avg 、 Sum sum
Scene : Inquiry language 、 Mathematics 、 English The maximum of these three subjects 、 minimum value 、 Average
POST http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "max_chinese" : { "max" : { "field" : "chinese" } }, "min_chinese" : { "min" : { "field" : "chinese" } }, "avg_chinese" : { "avg" : { "field" : "chinese" } }, "max_math": { "max" : { "field" : "math" } }, "min_math": { "min" : { "field" : "math" } }, "avg_math": { "avg" : { "field" : "math" } }, "max_english": { "max" : { "field" : "english" } }, "min_english": { "min" : { "field" : "english" } }, "avg_english": { "avg" : { "field" : "english" } } } }
The query result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "avg_english": { "value": 57.78366490546798 }, "max_chinese": { "value": 98 }, "min_chinese": { "value": 25 }, "min_math": { "value": 15 }, "max_english": { "value": 98 }, "avg_chinese": { "value": 59.353859695794505 }, "avg_math": { "value": 56.92907568735187 }, "min_english": { "value": 21 }, "max_math": { "value": 99 } } }
You can also query the total score of Chinese subjects , It is equivalent to sql Of sum Logic , Although it doesn't make any sense here :
POST http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "sum_chinese" : { "sum" : { "field" : "chinese" } } } }
2、 Count the number , It is equivalent to sql Of count Logic
Scene : Query the total number of all students , It's easy here count One The column is OK , For example, the field of Mathematics
POST http://localhost:9200/student_test1/_search?size=0 { "aggs": { "age_count": { "value_count": { "field": "math" } } } }
The return result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "age_count": { "value": 50084828 } } }
The total number between classes is :50084828 It's consistent with the amount of data we generated in Chapter 5
3、distinct polymerization , It is equivalent to sql Of count ( distinct )
Scene : How many kinds of value does the statistics language achievement have
POST http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "type_count" : { "cardinality" : { "field" : "chinese" } } } }
The return result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "type_count": { "value": 74 } } }
In terms of results , Only 74 A different score , Match with the rules of the fifth randomly generated data
4、 Statistical aggregation
Scene : Query language scores Total number 、 Maximum 、 minimum value 、 Average 、 Sum up, etc
POST http://localhost:9200/student_test1/_search?size=0 { "aggs": { "chinese_stats": { "stats": { "field": "chinese" } } } }
The return result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "chinese_stats": { "count": 50084828, "min": 25, "max": 98, "avg": 59.353859695794505, "sum": 2972727854 } } }
5、 Enhanced statistical aggregation , The query results are based on the above , Plus the variance and other statistical data
POST http://localhost:9200/student_test1/_search?size=0 { "aggs": { "chinese_stats": { "extended_stats": { "field": "chinese" } } } }
6、 Quantile aggregation Statistics
The default quantile is 1% 5% 25% 50% 75% 95% 99% 《= The concept of
The concept of quantile :25% The quantile of is 54, Less than or equal to 54 Of the total sample 25% , That is 54 This number will be at the bottom of 1/4 We're going to split it up .
POST http://localhost:9200/student_test1/_search?size=0 { "aggs": { "chinese_percents": { "percentiles": { "field": "chinese" } } } }
You can also customize the quantile :
POST http://localhost:9200/student_test1/_search?size=0 { "aggs": { "chinese_percents": { "percentiles": { "field": "chinese", "percents" : [10,20,30,40,50,60,70,80,90] } } } }
7、 Range aggregation Statistics
Scene : The language scores are less than 40 branch 、 Smaller than 50 branch 、 Smaller than 60 The percentage of points
POST http://localhost:9200/student_test1/_search?size=0 { "aggs": { "gge_perc_rank": { "percentile_ranks": { "field": "chinese", "values": [40,50,60] } } } }
The above results are less than 40, Smaller than 50, Smaller than 60 % of , The information we get is : 21.29% 36.09% 51.12% As you can see, it's a sequence that's close to arithmetic , It can be seen that the randomness of the test data is still very good .
The second part Other polymerization methods
1、Term polymerization
Scene : Want to know the students' Chinese achievement , The number of points on all fractions
POST http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "genres" : { "terms" : { "field" : "chinese" } } } }
This query will put the fields Chinese Aggregate , for example 87 To synthesize a group of ,88 To synthesize a group of , wait ;
But the default here is to sort by group size , And it doesn't show all the groups , Groups that are too small may be ignored , The results are as follows :
{ "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "genres": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 42560269, "buckets": [ { "key": 61, "doc_count": 752863 }, { "key": 68, "doc_count": 752835 }, { "key": 55, "doc_count": 752749 }, { "key": 59, "doc_count": 752444 }, { "key": 76, "doc_count": 752405 }, { "key": 74, "doc_count": 752309 }, { "key": 56, "doc_count": 752283 }, { "key": 49, "doc_count": 752273 }, { "key": 52, "doc_count": 752201 }, { "key": 50, "doc_count": 752197 } ] } } }
If you want to customize the filter criteria ,Term Aggregation can also be queried according to the following settings :
post http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "genres" : { "terms" : { "field" : "chinese", "size" : 100, // There may be 100 It's an unused score , We're going to show it all "order" : { "_count" : "asc" }, // Sort according to the number of groups "min_doc_count": 752200 // Filter conditions : The minimum number of groups is 752200 } } } }
The query result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "genres": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": 52, "doc_count": 752201 }, { "key": 49, "doc_count": 752273 }, { "key": 56, "doc_count": 752283 }, { "key": 74, "doc_count": 752309 }, { "key": 76, "doc_count": 752405 }, { "key": 59, "doc_count": 752444 }, { "key": 55, "doc_count": 752749 }, { "key": 68, "doc_count": 752835 }, { "key": 61, "doc_count": 752863 } ] } } }
2、Filter polymerization
Filter Aggregation is conditional filtering first , In the process of polymerization
Scene : Query the average score of students in South China University of technology ( First screen schools , And then we aggregate the scores )
{ "aggs" : { "scut_math_avg" : { "filter" : { "term": { "school": " South China University of Technology " } }, "aggs" : { "avg_price" : { "avg" : { "field" : "math" } } } } } }
The query result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "scut_math_avg": { "doc_count": 1854993, "avg_price": { "value": 56.93080027795253 } } } }
3、Filters Multiple aggregation
Scene : Check schools , Chinese 、 Mathematics 、 What is the average score of English , Multiple polymerization can be used , It may be a little slow , as follows
POST http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "messages" : { "filters" : { "filters" : { "school_1" : { "term" : { "school" : " South China University of Technology " }}, "school_2" : { "term" : { "school" : " Sun Yat sen University " }}, "school_3" : { "match" : { "school" : " Jinan University " }} } }, "aggs" : { "avg_chinese" : { "avg" : { "field" : "chinese" } }, "avg_math" : { "avg" : { "field" : "math" } } } } } }
So the result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "messages": { "buckets": { "school_1": { "doc_count": 1854993, "avg_chinese": { "value": 59.353236912484306 }, "avg_math": { "value": 56.93080027795253 } }, "school_2": { "doc_count": 1855016, "avg_chinese": { "value": 59.349129064115886 }, "avg_math": { "value": 56.93540918245449 } }, "school_3": { "doc_count": 44519876, "avg_chinese": { "value": 59.35397212247402 }, "avg_math": { "value": 56.92948502372289 } } } } } }
4、Range Range aggregation
Scene : Want to query the number of Chinese scores in each segment , You can query
POST http://localhost:9200/student_test1/_search?size=0
{ "aggs" : { "chinese_ranges" : { "range" : { "field" : "chinese", "ranges" : [ { "to" : 60 }, { "from" : 60, "to" : 75 }, { "from" : 75, "to" : 85 }, { "from" : 85 } ] } } } }
The query result is :
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "chinese_ranges": { "buckets": [ { "key": "*-60.0", "to": 60, "doc_count": 25096839 }, { "key": "60.0-75.0", "from": 60, "to": 75, "doc_count": 11278543 }, { "key": "75.0-85.0", "from": 75, "to": 85, "doc_count": 7424634 }, { "key": "85.0-*", "from": 85, "doc_count": 6284812 } ] } } }
The group names of the returned results are *-60.0 60.0-75.0 75.0-85.0 85.0-*
If we don't want a group name like this , You can customize the group name , for example :
POST http://localhost:9200/student_test1/_search?size=0 { "aggs" : { "chinese_ranges" : { "range" : { "field" : "chinese", "keyed" : true, "ranges" : [ { "key" : " fail, ", "to" : 60 }, { "key" : " pass ", "from" : 60, "to" : 75 }, { "key" : " good ", "from" : 75, "to" : 85 }, { "key" : " Excellent ", "from" : 85 } ] } } } }
The query result will be :
{ "took": 1675, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 10000, "relation": "gte" }, "max_score": null, "hits": [] }, "aggregations": { "chinese_ranges": { "buckets": { " fail, ": { "to": 60, "doc_count": 25096839 }, " pass ": { "from": 60, "to": 75, "doc_count": 11278543 }, " good ": { "from": 75, "to": 85, "doc_count": 7424634 }, " Excellent ": { "from": 85, "doc_count": 6284812 } } } } }
There are all kinds of other 、 Complex aggregate queries , All of them can check information online , It even supports some calculation methods of recommendation system , For example, the concept of matrix and so on .
You can also refer to https://blog.csdn.net/alex_xfboy/article/details/8610
版权声明
本文为[itread01]所创,转载请带上原文链接,感谢
边栏推荐
猜你喜欢
A course on word embedding
Brief introduction of TF flags
C + + and C + + programmers are about to be eliminated from the market
What are Devops
代码生成器插件与Creator预制体文件解析
Mac installation hanlp, and win installation and use
FastThreadLocal 是什么鬼?吊打 ThreadLocal 的存在!!
Brief introduction and advantages and disadvantages of deepwalk model
给字节的学姐讲如何准备“系统设计面试”
Flink的DataSource三部曲之一:直接API
随机推荐
Natural language processing - BM25 commonly used in search
Discussion on the development practice of aspnetcore, a cross platform framework
TensorFlow中的Tensor是什么?
What are Devops
一篇文章带你了解CSS 渐变知识
Using NLP and ml to extract and construct web data
Who says cat can't do link tracking? Stand up for me
百万年薪,国内工作6年的前辈想和你分享这四点
How to use parameters in ES6
WeihanLi.Npoi 1.11.0/1.12.0 Release Notes
Gather in Beijing! The countdown to openi 2020
5.4 static resource mapping
C + + and C + + programmers are about to be eliminated from the market
開源一套極簡的前後端分離專案腳手架
[Xinge education] poor learning host computer series -- building step 7 Simulation Environment
The difference between gbdt and XGB, and the mathematical derivation of gradient descent method and Newton method
Basic principle and application of iptables
DC-1靶機
What if the front end doesn't use spa? - Hacker News
Jmeter——ForEach Controller&Loop Controller