当前位置：网站首页>Mapping settings in elk (8) es
Mapping settings in elk (8) es
2022-07-02 04:54:00 【wx5ba7ab4695f27】
List of articles
- effect
- Get index mapping
- Customize Mapping
- Dynamic Mapping
- copy_to
- Index attribute
- Core data type
- String type ：text、keyword
- Binary type ：binary
- Complex data type
- Geographic location data type
- Special type
- Multi field features multi-fields
- Dynamic Mapping
effect
Define the structure of tables in the database , adopt mapping To control the setting of index storage data
a. Definition Index The field name below （Field Name）
b. Define the type of field , For example, numerical type 、 String type 、 Boolean, etc
c. Define the configuration related to inverted index , such as documentId、 Record position、 Grade
Get index mapping
When not configured , Automatically created mapping
Inject test data
POST test/doc/1
{
"name":"zyd",
"age":16
}
- 1.
- 2.
- 3.
- 4.
- 5.
request ：
GET test/_mapping
- 1.
return
{
"test": { # The index name
"mappings": { #mapping Set up
"doc": { #type name
"properties": { # Field properties
"age": {
"type": "long" # Field type , String default type
},
"name": {
"type": "text",
"fields": { # Subfield attribute settings
"keyword": { # Participle type （ No participle ）
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
Customize Mapping
request
PUT my_index
{
"mappings":{
"doc":{ # Type the name
"dynamic":false,
"properties":{
"title":{
"type":"text" # Field type
},
"name":{
"type":"keyword"
},
"age":{
"type":"integer"
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
response
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "my_index"
}
- 1.
- 2.
- 3.
- 4.
- 5.
Dynamic Mapping
es rely on json Document field type to realize automatic recognition of field type , Types of support
JSON type | es type |
null | Ignore |
boolean | boolean |
Floating point type | float |
Integers | long |
object | object |
array | By the first non null The type of value determines |
string | If the matching date is, it is set to data type （ Default on ） If the match is a number, set it to float or long type （ Off by default ） Set to text type , With keyword Sub fields of |
Be careful ：
mapping Once the field type in is set , No modification
reason ：Lucene The inverted index is not allowed to be modified after it is generated ( Increase of efficiency )
If you want to change the type of the field , Need to re index , Then I do reindex operation
dynamic Set up
a. true： Allow automatic addition of fields （ Default configuration ）
b. False： Automatic addition of fields is not allowed , But the document can be written normally , Unable to query fields
c. strict： The document cannot be written to （ An error will be reported if writing ）
PUT my_index1
{
"mappings":{
"doc":{
"dynamic":false,
"properties":{
"user":{
"properties":{
"name":{
"type":"text"
},"social_networks":{
"dynamic":true,
"properties":{}
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
copy_to
Copy the value of the field to , Realization _all The role of , Will not appear in _source in , Only for searching
PUT my_index3
{
"mappings": {
"my_type": {
"properties": {
"first_name": {
"type":"keyword",
"copy_to":"full_name"
},
"last_name": {
"type":"keyword",
"copy_to":"full_name"
},
"full_name": {
"type":"text",
"fielddata":true
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
insert data
PUT my_index3/my_type/1
{
"first_name": "John",
"last_name": "Smith"
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
obtain
GET my_index3/_search
{
"query": {
"match": {
"full_name": {
"query":"John Smith",
"operator":"and"
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
result
{
"took": 215,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.5753642,
"hits": [
{
"_index": "my_index3",
"_type": "my_type",
"_id": "1",
"_score": 0.5753642,
"_source": {
"first_name": "John",
"last_name": "Smith"
}
}
]
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
Index attribute
Index attribute , Controls whether the current field is indexed , The default is true, The record index ,false Don't record , That is, you cannot search , such as ： cell-phone number 、 Sensitive information such as ID number , Do not want to be retrieved
1、 establish mapping
PUT my_index
{
"mappings": {
"doc":{
"properties": {
"cookie":{
"type":"text",
"index": false
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
1、 create documents
PUT my_index/doc/1
{
"cookie":"123",
"name":"home"
}
- 1.
- 2.
- 3.
- 4.
- 5.
Inquire about
GET my_index/_search
{
"query": {
"match": {
"cookie":"123"
}
}
}
# Report errors
GET my_index/_search
{
"query": {
"match": {
"name":"home"
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
Index_options Used to control the contents of inverted index records , There are the following 4 Middle configuration
docs： Record only docid
freqs： Record docid and term frequencies（ Word frequency ）
position： Record docid、term frequencies、term position
Offsets： Record docid、term frequencies、term position、character offsets
text The default configuration of the type is position, Its default is that docs
The more you record , The more space it takes
data type
Core data type
String type ：text、keyword
Numerical type ：long、integer、short、byte、double、float、half_float、scaled_float
The date type ：date
Boolean type ：boolean
Binary type ：binary
Range type ：integer_range、float_range、long_range、double_range、date_range
Complex data type
An array type ：array
object type ：object
Nested Type ：nested object
Geographic location data type
geo_point( spot )、geo_shape( shape )
Special type
Record IP Address ip
Realize automatic completion completion
Record the number of words ：token_count
Record string hash Value breast milk murmur3
Multi field features multi-fields
Allow different configurations for the same field , For example, participle , For example, the implementation of Pinyin search for people's names , You just need to add a sub field in the person name as pinyin that will do
1、 establish mapping
PUT my_index1
{
"mappings": {
"doc":{
"properties":{
"username":{
"type": "text",
"fields": {
"pinyin":{
"type": "text"
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
create documents
PUT my_index1/doc/1
{
"username":"haha heihei"
}
- 1.
- 2.
- 3.
- 4.
Inquire about
GET my_index1/_search
{
"query": {
"match": {
"username.pinyin": "haha"
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
Dynamic Mapping
es Can automatically identify document field type , So as to reduce the cost of users
PUT /test_index/doc/1
{
"username":"alfred",
"age":1
}
- 1.
- 2.
- 3.
- 4.
- 5.
{
"test_index": {
"mappings": {
"doc": {
"properties": {
"age": {
"type": "long"
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
age Automatically identified as long type ,username Identified as text type
PUT test_index/doc/1
{
"username":"samualz",
"age":14,
"birth":"1991-12-15",
"year":18,
"tags":["boy","fashion"],
"money":"100.1"
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
{
"test_index": {
"mappings": {
"doc": {
"properties": {
"age": {
"type": "long"
},
"birth": {
"type": "date"
},
"money": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"tags": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"year": {
"type": "long"
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
The automatic identification of date can configure the date format , To meet all kinds of needs
1、 Custom date recognition format
PUT my_index
{
"mappings":{
"doc":{
"dynamic_date_formats": ["yyyy-MM-dd","yyyy/MM/dd"]
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
Turn off automatic date recognition
PUT my_index
{
"mappings": {
"doc": {
"date_detection": false
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
When a string is a number , By default, it is not automatically recognized as an integer , Because it makes perfect sense to have numbers in a string
Numeric_datection You can turn on the automatic recognition of numbers in strings
PUT my_index
{
"mappings":{
"doc":{
"numeric_datection": true
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
边栏推荐
- Let正版短信测压开源源码
- 关于Steam 教育的知识整理
- Several methods of capturing packets under CS framework
- Introduction to Luogu 3 [circular structure] problem list solution
- Steam教育的实际问题解决能力
- Learn BeanShell before you dare to say you know JMeter
- 2022-003arts: recursive routine of binary tree
- 社交媒体搜索引擎优化及其重要性
- What data does the main account of Zhengda Meiou 4 pay attention to?
- 数据库问题汇总
猜你喜欢
List of common bugs in software testing
Gin framework learning code
Alibaba cloud polkit pkexec local rights lifting vulnerability
LeetCode-对链表进行插入排序
Promise all()
数学问题（数论）试除法做质数的判断、分解质因数，筛质数
Pit encountered in win11 pytorch GPU installation
Record the bug of unity 2020.3.31f1 once
idea自動導包和自動删包設置
Federal learning: dividing non IID samples according to Dirichlet distribution
随机推荐
Keil compilation code of CY7C68013A
洛谷入门3【循环结构】题单题解
DJB Hash
社交媒体搜索引擎优化及其重要性
Binary tree problem solving (2)
Mysql database learning
Detailed process of DC-1 range construction and penetration practice (DC range Series)
Knowledge arrangement about steam Education
06 decorator mode
Lay the foundation for children's programming to become a basic discipline
Federal learning: dividing non IID samples according to Dirichlet distribution
[bus interface] Axi interface
Exposure X8 Standard Version picture post filter PS, LR and other software plug-ins
2022阿里巴巴全球数学竞赛 第4题 虎虎生威（盲盒问题、集卡问题）解决思路
Comp 250 parsing
DC-1靶场搭建及渗透实战详细过程（DC靶场系列）
Online incremental migration of DM database
Alibaba cloud polkit pkexec local rights lifting vulnerability
Mouse events in JS
JS interview collection test question 1