当前位置：网站首页>Elasticsearch Part 6: aggregate statistical query
Elasticsearch Part 6: aggregate statistical query
2020-11-06 20:10:00 【itread01】
There has been no record before Elasticsearch Aggregate queries or other complex queries . Take notes in this article , To facilitate testing , The index data is still the test index library generated in Article 5 db_student_test , The nickname is student_test
The first part Basic aggregation
1、 Maximum max、 minimum value min、 Average avg 、 Sum sum
Scene ： Inquiry language 、 Mathematics 、 English The maximum of these three subjects 、 minimum value 、 Average
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"max_chinese" : { "max" : { "field" : "chinese" } },
"min_chinese" : { "min" : { "field" : "chinese" } },
"avg_chinese" : { "avg" : { "field" : "chinese" } },
"max_math": { "max" : { "field" : "math" } },
"min_math": { "min" : { "field" : "math" } },
"avg_math": { "avg" : { "field" : "math" } },
"max_english": { "max" : { "field" : "english" } },
"min_english": { "min" : { "field" : "english" } },
"avg_english": { "avg" : { "field" : "english" } }
}
}
The query result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"avg_english": {
"value": 57.78366490546798
},
"max_chinese": {
"value": 98
},
"min_chinese": {
"value": 25
},
"min_math": {
"value": 15
},
"max_english": {
"value": 98
},
"avg_chinese": {
"value": 59.353859695794505
},
"avg_math": {
"value": 56.92907568735187
},
"min_english": {
"value": 21
},
"max_math": {
"value": 99
}
}
}
You can also query the total score of Chinese subjects , It is equivalent to sql Of sum Logic , Although it doesn't make any sense here ：
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"sum_chinese" : { "sum" : { "field" : "chinese" } }
}
}
2、 Count the number , It is equivalent to sql Of count Logic
Scene ： Query the total number of all students , It's easy here count One The column is OK , For example, the field of Mathematics
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs": {
"age_count": {
"value_count": {
"field": "math"
}
}
}
}
The return result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"age_count": {
"value": 50084828
}
}
}
The total number between classes is ：50084828 It's consistent with the amount of data we generated in Chapter 5
3、distinct polymerization , It is equivalent to sql Of count ( distinct )
Scene ： How many kinds of value does the statistics language achievement have
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"type_count" : {
"cardinality" : {
"field" : "chinese"
}
}
}
}
The return result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"type_count": {
"value": 74
}
}
}
In terms of results , Only 74 A different score , Match with the rules of the fifth randomly generated data
4、 Statistical aggregation
Scene ： Query language scores Total number 、 Maximum 、 minimum value 、 Average 、 Sum up, etc
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs": {
"chinese_stats": {
"stats": {
"field": "chinese"
}
}
}
}
The return result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"chinese_stats": {
"count": 50084828,
"min": 25,
"max": 98,
"avg": 59.353859695794505,
"sum": 2972727854
}
}
}
5、 Enhanced statistical aggregation , The query results are based on the above , Plus the variance and other statistical data
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs": {
"chinese_stats": {
"extended_stats": {
"field": "chinese"
}
}
}
}
6、 Quantile aggregation Statistics
The default quantile is 1% 5% 25% 50% 75% 95% 99% 《= The concept of
The concept of quantile ：25% The quantile of is 54, Less than or equal to 54 Of the total sample 25% , That is 54 This number will be at the bottom of 1/4 We're going to split it up .
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs": {
"chinese_percents": {
"percentiles": {
"field": "chinese"
}
}
}
}
You can also customize the quantile ：
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs": {
"chinese_percents": {
"percentiles": {
"field": "chinese",
"percents" : [10,20,30,40,50,60,70,80,90]
}
}
}
}
7、 Range aggregation Statistics
Scene ： The language scores are less than 40 branch 、 Smaller than 50 branch 、 Smaller than 60 The percentage of points
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs": {
"gge_perc_rank": {
"percentile_ranks": {
"field": "chinese",
"values": [40,50,60]
}
}
}
}
The above results are less than 40, Smaller than 50, Smaller than 60 % of , The information we get is ： 21.29% 36.09% 51.12% As you can see, it's a sequence that's close to arithmetic , It can be seen that the randomness of the test data is still very good .
The second part Other polymerization methods
1、Term polymerization
Scene ： Want to know the students' Chinese achievement , The number of points on all fractions
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"genres" : {
"terms" : {
"field" : "chinese"
}
}
}
}
This query will put the fields Chinese Aggregate , for example 87 To synthesize a group of ,88 To synthesize a group of , wait ;
But the default here is to sort by group size , And it doesn't show all the groups , Groups that are too small may be ignored , The results are as follows ：
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"genres": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 42560269,
"buckets": [
{
"key": 61,
"doc_count": 752863
},
{
"key": 68,
"doc_count": 752835
},
{
"key": 55,
"doc_count": 752749
},
{
"key": 59,
"doc_count": 752444
},
{
"key": 76,
"doc_count": 752405
},
{
"key": 74,
"doc_count": 752309
},
{
"key": 56,
"doc_count": 752283
},
{
"key": 49,
"doc_count": 752273
},
{
"key": 52,
"doc_count": 752201
},
{
"key": 50,
"doc_count": 752197
}
]
}
}
}
If you want to customize the filter criteria ,Term Aggregation can also be queried according to the following settings ：
post http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"genres" : {
"terms" : {
"field" : "chinese",
"size" : 100, // There may be 100 It's an unused score , We're going to show it all
"order" : { "_count" : "asc" }, // Sort according to the number of groups
"min_doc_count": 752200 // Filter conditions ： The minimum number of groups is 752200
}
}
}
}
The query result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"genres": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 52,
"doc_count": 752201
},
{
"key": 49,
"doc_count": 752273
},
{
"key": 56,
"doc_count": 752283
},
{
"key": 74,
"doc_count": 752309
},
{
"key": 76,
"doc_count": 752405
},
{
"key": 59,
"doc_count": 752444
},
{
"key": 55,
"doc_count": 752749
},
{
"key": 68,
"doc_count": 752835
},
{
"key": 61,
"doc_count": 752863
}
]
}
}
}
2、Filter polymerization
Filter Aggregation is conditional filtering first , In the process of polymerization
Scene ： Query the average score of students in South China University of technology （ First screen schools , And then we aggregate the scores ）
{
"aggs" : {
"scut_math_avg" : {
"filter" : { "term": { "school": " South China University of Technology " } },
"aggs" : {
"avg_price" : { "avg" : { "field" : "math" } }
}
}
}
}
The query result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"scut_math_avg": {
"doc_count": 1854993,
"avg_price": {
"value": 56.93080027795253
}
}
}
}
3、Filters Multiple aggregation
Scene ： Check schools , Chinese 、 Mathematics 、 What is the average score of English , Multiple polymerization can be used , It may be a little slow , as follows
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"messages" : {
"filters" : {
"filters" : {
"school_1" : { "term" : { "school" : " South China University of Technology " }},
"school_2" : { "term" : { "school" : " Sun Yat sen University " }},
"school_3" : { "match" : { "school" : " Jinan University " }}
}
},
"aggs" : {
"avg_chinese" : { "avg" : { "field" : "chinese" } },
"avg_math" : { "avg" : { "field" : "math" } }
}
}
}
}
So the result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"messages": {
"buckets": {
"school_1": {
"doc_count": 1854993,
"avg_chinese": {
"value": 59.353236912484306
},
"avg_math": {
"value": 56.93080027795253
}
},
"school_2": {
"doc_count": 1855016,
"avg_chinese": {
"value": 59.349129064115886
},
"avg_math": {
"value": 56.93540918245449
}
},
"school_3": {
"doc_count": 44519876,
"avg_chinese": {
"value": 59.35397212247402
},
"avg_math": {
"value": 56.92948502372289
}
}
}
}
}
}
4、Range Range aggregation
Scene ： Want to query the number of Chinese scores in each segment , You can query
POST http://localhost:9200/student_test1/_search?size=0
{ "aggs" : { "chinese_ranges" : { "range" : { "field" : "chinese", "ranges" : [ { "to" : 60 }, { "from" : 60, "to" : 75 }, { "from" : 75, "to" : 85 }, { "from" : 85 } ] } } } }
The query result is ：
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"chinese_ranges": {
"buckets": [
{
"key": "*-60.0",
"to": 60,
"doc_count": 25096839
},
{
"key": "60.0-75.0",
"from": 60,
"to": 75,
"doc_count": 11278543
},
{
"key": "75.0-85.0",
"from": 75,
"to": 85,
"doc_count": 7424634
},
{
"key": "85.0-*",
"from": 85,
"doc_count": 6284812
}
]
}
}
}
The group names of the returned results are *-60.0 60.0-75.0 75.0-85.0 85.0-*
If we don't want a group name like this , You can customize the group name , for example ：
POST http://localhost:9200/student_test1/_search?size=0
{
"aggs" : {
"chinese_ranges" : {
"range" : {
"field" : "chinese",
"keyed" : true,
"ranges" : [
{ "key" : " fail, ", "to" : 60 },
{ "key" : " pass ", "from" : 60, "to" : 75 },
{ "key" : " good ", "from" : 75, "to" : 85 },
{ "key" : " Excellent ", "from" : 85 }
]
}
}
}
}
The query result will be ：
{
"took": 1675,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"chinese_ranges": {
"buckets": {
" fail, ": {
"to": 60,
"doc_count": 25096839
},
" pass ": {
"from": 60,
"to": 75,
"doc_count": 11278543
},
" good ": {
"from": 75,
"to": 85,
"doc_count": 7424634
},
" Excellent ": {
"from": 85,
"doc_count": 6284812
}
}
}
}
}
There are all kinds of other 、 Complex aggregate queries , All of them can check information online , It even supports some calculation methods of recommendation system , For example, the concept of matrix and so on .
You can also refer to https://blog.csdn.net/alex_xfboy/article/details/8610
版权声明
本文为[itread01]所创，转载请带上原文链接，感谢
边栏推荐
- 視覺滾動[反差美]
- How to hide part of barcode text in barcode generation software
- GUI engine evaluation index
- 【應用程式見解 Application Insights】Application Insights 使用 Application Maps 構建請求鏈路檢視
- Introduction to X Window System
- （1） ASP.NET Introduction to core3.1 Ocelot
- What knowledge do Python automated testing learn?
- Humor: hacker programming is actually similar to machine learning!
- Natural language processing - BM25 commonly used in search
- 一篇文章带你了解CSS3 背景知识
猜你喜欢
前端基础牢记的一些操作-Github仓库管理
Python filtering sensitive word records
Pattern matching: The gestalt approach一种序列的文本相似度方法
The importance of big data application is reflected in all aspects
用一个例子理解JS函数的底层处理机制
Named entity recognition in natural language processing: tanford core LP ner (1)
FastThreadLocal 是什么鬼？吊打 ThreadLocal 的存在！！
Interface pressure test: installation, use and instruction of siege pressure test
一篇文章教会你使用Python网络爬虫下载酷狗音乐
Wow, elasticsearch multi field weight sorting can play like this
随机推荐
Brief introduction of TF flags
use Asponse.Words Working with word templates
The road of C + + Learning: from introduction to mastery
C語言I部落格作業03
Analysis of ThreadLocal principle
事件监听问题
Multi classification of unbalanced text using AWS sagemaker blazingtext
Vue.js Mobile end left slide delete component
What are manufacturing and new automation technologies?
Python基础数据类型——tuple浅析
Custom function form of pychar shortcut key
Python download module to accelerate the implementation of recording
ES6学习笔记（四）：教你轻松搞懂ES6的新增语法
DC-1靶機
Chainlink brings us election results into blockchain everipedia
Live broadcast preview | micro service architecture Learning Series live broadcast phase 3
MeterSphere开发者手册
每个大火的“线上狼人杀”平台，都离不开这个新功能
Analysis of partial source codes of qthread
C＃和C/C++混合编程系列5-内存管理之GC协同