当前位置：网站首页>Mapping settings in elk (8) es
Mapping settings in elk (8) es
2022-07-02 04:54:00 【wx5ba7ab4695f27】
List of articles
- effect
- Get index mapping
- Customize Mapping
- Dynamic Mapping
- copy_to
- Index attribute
- Core data type
- String type ：text、keyword
- Binary type ：binary
- Complex data type
- Geographic location data type
- Special type
- Multi field features multi-fields
- Dynamic Mapping
effect
Define the structure of tables in the database , adopt mapping To control the setting of index storage data
a. Definition Index The field name below （Field Name）
b. Define the type of field , For example, numerical type 、 String type 、 Boolean, etc
c. Define the configuration related to inverted index , such as documentId、 Record position、 Grade
Get index mapping
When not configured , Automatically created mapping
Inject test data
POST test/doc/1
{
"name":"zyd",
"age":16
}
- 1.
- 2.
- 3.
- 4.
- 5.
request ：
GET test/_mapping
- 1.
return
{
"test": { # The index name
"mappings": { #mapping Set up
"doc": { #type name
"properties": { # Field properties
"age": {
"type": "long" # Field type , String default type
},
"name": {
"type": "text",
"fields": { # Subfield attribute settings
"keyword": { # Participle type （ No participle ）
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
Customize Mapping
request
PUT my_index
{
"mappings":{
"doc":{ # Type the name
"dynamic":false,
"properties":{
"title":{
"type":"text" # Field type
},
"name":{
"type":"keyword"
},
"age":{
"type":"integer"
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
response
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "my_index"
}
- 1.
- 2.
- 3.
- 4.
- 5.
Dynamic Mapping
es rely on json Document field type to realize automatic recognition of field type , Types of support
JSON type | es type |
null | Ignore |
boolean | boolean |
Floating point type | float |
Integers | long |
object | object |
array | By the first non null The type of value determines |
string | If the matching date is, it is set to data type （ Default on ） If the match is a number, set it to float or long type （ Off by default ） Set to text type , With keyword Sub fields of |
Be careful ：
mapping Once the field type in is set , No modification
reason ：Lucene The inverted index is not allowed to be modified after it is generated ( Increase of efficiency )
If you want to change the type of the field , Need to re index , Then I do reindex operation
dynamic Set up
a. true： Allow automatic addition of fields （ Default configuration ）
b. False： Automatic addition of fields is not allowed , But the document can be written normally , Unable to query fields
c. strict： The document cannot be written to （ An error will be reported if writing ）
PUT my_index1
{
"mappings":{
"doc":{
"dynamic":false,
"properties":{
"user":{
"properties":{
"name":{
"type":"text"
},"social_networks":{
"dynamic":true,
"properties":{}
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
copy_to
Copy the value of the field to , Realization _all The role of , Will not appear in _source in , Only for searching
PUT my_index3
{
"mappings": {
"my_type": {
"properties": {
"first_name": {
"type":"keyword",
"copy_to":"full_name"
},
"last_name": {
"type":"keyword",
"copy_to":"full_name"
},
"full_name": {
"type":"text",
"fielddata":true
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
insert data
PUT my_index3/my_type/1
{
"first_name": "John",
"last_name": "Smith"
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
obtain
GET my_index3/_search
{
"query": {
"match": {
"full_name": {
"query":"John Smith",
"operator":"and"
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
result
{
"took": 215,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.5753642,
"hits": [
{
"_index": "my_index3",
"_type": "my_type",
"_id": "1",
"_score": 0.5753642,
"_source": {
"first_name": "John",
"last_name": "Smith"
}
}
]
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
Index attribute
Index attribute , Controls whether the current field is indexed , The default is true, The record index ,false Don't record , That is, you cannot search , such as ： cell-phone number 、 Sensitive information such as ID number , Do not want to be retrieved
1、 establish mapping
PUT my_index
{
"mappings": {
"doc":{
"properties": {
"cookie":{
"type":"text",
"index": false
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
1、 create documents
PUT my_index/doc/1
{
"cookie":"123",
"name":"home"
}
- 1.
- 2.
- 3.
- 4.
- 5.
Inquire about
GET my_index/_search
{
"query": {
"match": {
"cookie":"123"
}
}
}
# Report errors
GET my_index/_search
{
"query": {
"match": {
"name":"home"
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
Index_options Used to control the contents of inverted index records , There are the following 4 Middle configuration
docs： Record only docid
freqs： Record docid and term frequencies（ Word frequency ）
position： Record docid、term frequencies、term position
Offsets： Record docid、term frequencies、term position、character offsets
text The default configuration of the type is position, Its default is that docs
The more you record , The more space it takes
data type
Core data type
String type ：text、keyword
Numerical type ：long、integer、short、byte、double、float、half_float、scaled_float
The date type ：date
Boolean type ：boolean
Binary type ：binary
Range type ：integer_range、float_range、long_range、double_range、date_range
Complex data type
An array type ：array
object type ：object
Nested Type ：nested object
Geographic location data type
geo_point( spot )、geo_shape( shape )
Special type
Record IP Address ip
Realize automatic completion completion
Record the number of words ：token_count
Record string hash Value breast milk murmur3
Multi field features multi-fields
Allow different configurations for the same field , For example, participle , For example, the implementation of Pinyin search for people's names , You just need to add a sub field in the person name as pinyin that will do
1、 establish mapping
PUT my_index1
{
"mappings": {
"doc":{
"properties":{
"username":{
"type": "text",
"fields": {
"pinyin":{
"type": "text"
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
create documents
PUT my_index1/doc/1
{
"username":"haha heihei"
}
- 1.
- 2.
- 3.
- 4.
Inquire about
GET my_index1/_search
{
"query": {
"match": {
"username.pinyin": "haha"
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
Dynamic Mapping
es Can automatically identify document field type , So as to reduce the cost of users
PUT /test_index/doc/1
{
"username":"alfred",
"age":1
}
- 1.
- 2.
- 3.
- 4.
- 5.
{
"test_index": {
"mappings": {
"doc": {
"properties": {
"age": {
"type": "long"
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
age Automatically identified as long type ,username Identified as text type
PUT test_index/doc/1
{
"username":"samualz",
"age":14,
"birth":"1991-12-15",
"year":18,
"tags":["boy","fashion"],
"money":"100.1"
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
{
"test_index": {
"mappings": {
"doc": {
"properties": {
"age": {
"type": "long"
},
"birth": {
"type": "date"
},
"money": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"tags": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"year": {
"type": "long"
}
}
}
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
The automatic identification of date can configure the date format , To meet all kinds of needs
1、 Custom date recognition format
PUT my_index
{
"mappings":{
"doc":{
"dynamic_date_formats": ["yyyy-MM-dd","yyyy/MM/dd"]
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
Turn off automatic date recognition
PUT my_index
{
"mappings": {
"doc": {
"date_detection": false
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
When a string is a number , By default, it is not automatically recognized as an integer , Because it makes perfect sense to have numbers in a string
Numeric_datection You can turn on the automatic recognition of numbers in strings
PUT my_index
{
"mappings":{
"doc":{
"numeric_datection": true
}
}
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
边栏推荐
- Cache consistency solution - how to ensure the consistency between the cache and the data in the database when changing data
- One click generation and conversion of markdown directory to word format
- VMware installation win10 reports an error: operating system not found
- C - derived classes and constructors
- Research on the security of ognl and El expressions and memory horse
- Oracle和MySQL的基本区别（入门级）
- 正大美欧4的主账户关注什么数据？
- One step implementation of yolox helmet detection (combined with oak intelligent depth camera)
- Mysql database learning
- 记录一次Unity 2020.3.31f1的bug
猜你喜欢
Mathematical knowledge (Euler function)
Vmware安装win10报错：operating system not found
记录一次Unity 2020.3.31f1的bug
My first experience of shadowless cloud computer
DC-1靶场搭建及渗透实战详细过程（DC靶场系列）
Summary of database problems
ansible安装与使用
![Learn AI safety monitoring project from zero [attach detailed code]](/img/a9/cb93f349229e86cbb05ad196ae9553.jpg)
Learn AI safety monitoring project from zero [attach detailed code]
Learn what definitelytyped is through the typescript development environment of SAP ui5
CorelDRAW graphics suite2022 free graphic design software
随机推荐
Leetcode merge sort linked list
Go Chan's underlying principles
Steam教育的实际问题解决能力
Exposure X8 Standard Version picture post filter PS, LR and other software plug-ins
[high speed bus] Introduction to jesd204b
idea自动导包和自动删包设置
Mysql database learning
Let正版短信测压开源源码
【ClickHouse】How to create index for Map Type Column or one key of it?
Idea automatic package import and automatic package deletion settings
win10 磁盘管理 压缩卷 无法启动问题
Idea autoguide package and autodelete package Settings
Orthogonal test method and function diagram method for test case design
Solution: the agent throws an exception error
MySQL table insert Chinese change? Solution to the problem of No
Design and implementation of general interface open platform - (44) log processing of API services
Leetcode- insert and sort the linked list
oracle 存储过程与job任务设置
Summary of MySQL key challenges (2)
06 decorator mode