elastic_ L04_ introduction. md

1、 Basic concepts

1.1 file

Elasticsearch It's document oriented , Documents are sequenced into JSON Format , Save in Elasticseach in .
Metadata of the document , Information used to label documents . Such as _index,_type,_id,_source,_version,_score.

# View the total number of documents indexed 
GET kibana_sample_data_ecommerce/_count

# See the specific fields 
GET /_cat/indices/kibana*?pri&v&h=health,index,pri,rep,docs.count,mt

#How much memory is used per index
GET /_cat/indices?v&h=i,tm&s=tm:desc

1.2 Indexes

Index It's a container for documents .Index Presenting the concept of logical space ;Shard Embody the concept of physical space , The data in the index is scattered in Shard in .

In the index Mapping Define the type of document field .
In the index Setting Define different data distributions .

# see indices
GET /_cat/indices/kibana*?v&s=index

1.3 node

Node is a Elasticsearch example
After each node starts , The default is a Master eligible node
Master-eligible Nodes can participate in the main selection process , Become Master node
Each node saves the status of the cluster , Only Master Nodes can modify the status information of the cluster

• Data Node
  Nodes that can hold data . Be responsible for saving segment data .
• Coordinating Node
  To accept Client Request , Say that the request is distributed to the appropriate node , Finally, bring the results together
  By default, each node plays Coordinating Node Responsibility for
• Other nodes （Hot & Warm Node、Machine Learning Node）

GET /_cat/nodes?v
GET /_cat/nodes?v&h=id,ip,port,v,m

CAT Nodes API

1.4 Fragmentation

• The primary shard
  Data can be distributed to all nodes in the cluster
  The number of primary partitions is locked when the index is created , Subsequent modifications are not allowed , Unless Reindex, Solve the problem of data horizontal expansion .
• copy
  A copy of the main slice , It can be adjusted dynamically , Solve the problem of high data availability .

GET _cat/shards
GET _cat/shards?h=index,shard,prirep,state,unassigned.reason

PUT /blogs
{
  "settings": {
     "number_of_shards": 3, "number_of_replicas": 1
  }
}

1.5 Reverse order

Word to document Id The relationship between
The reverse sort consists of two parts ： Word dictionary , Inverted list - Consists of inverted index entries
The inverted index entry contains ： file ID, Word frequency TF, Location , The offset

POST _analyze
{
  "analyzer": "standard",
  "text": "Elasticsearch Server"
}
resp:
{
  "tokens" : [
    {
      "token" : "elasticsearch",
      "start_offset" : 0,
      "end_offset" : 13,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "server",
      "start_offset" : 14,
      "end_offset" : 20,
      "type" : "<ALPHANUM>",
      "position" : 1
    }
  ]
}

Inverted index

1.6 Mapping

• What is? Mapping?
  Define the name of the field in the index 、 data type 、 Field , Related configurations of reverse sorting
  Will be able to JSON The document maps to Lucene Data format required .

• What is? Dynamic Mapping?
  No need to define manually Mapping,ES Automatically according to the document information , Figure out the type of field .

Display parameter settings

# Written document , see  Mapping
PUT mapping_test/_doc/1
{
  "firstName":"Chan",
  "lastName": "Jackie",
  "loginDate":"2018-07-24T10:29:48.103Z"
}

# see  Mapping file 
GET mapping_test/_mapping

#Delete index
DELETE mapping_test

#dynamic mapping, Infer the type of field 
PUT mapping_test/_doc/1
{
    "uid" : "123",
    "isVip" : false,
    "isAdmin": "true",
    "age":19,
    "heigh":180
}

# see  Dynamic
GET mapping_test/_mapping


# Default Mapping Support dynamic, Add new fields to the written document 
PUT dynamic_mapping_test/_doc/1
{
  "newField":"someValue"
}

# This field can be searched , The data is in _source It appears that 
POST dynamic_mapping_test/_search
{
  "query":{
    "match":{
      "newField":"someValue"
    }
  }
}

# It is amended as follows dynamic false
PUT dynamic_mapping_test/_mapping
{
  "dynamic": false
}

# newly added  anotherField
PUT dynamic_mapping_test/_doc/10
{
  "anotherField":"someValue"
}


# This field cannot be searched , because dynamic Has been set to false
POST dynamic_mapping_test/_search
{
  "query":{
    "match":{
      "anotherField":"someValue"
    }
  }
}

get dynamic_mapping_test/_doc/10

# It is amended as follows strict
PUT dynamic_mapping_test/_mapping
{
  "dynamic": "strict"
}



# Error writing data ,HTTP Code 400
PUT dynamic_mapping_test/_doc/12
{
  "lastField":"value"
}

DELETE dynamic_mapping_test

1.7 Index Tempplate

Help you set Mappings and Settings, And according to certain rules , Automatically match to the newly created index

Index Templates

2、 file CRUD

Document API

3、 Word segmentation through the analyzer

3.1 What is? Analysis

participle , It is the process of converting text into a series of words

# Use char filter Replace 
POST _analyze
{
  "tokenizer": "standard",
  "char_filter": [
      {
        "type" : "mapping",
        "mappings" : [ "- => _"]
      }
    ],
  "text": "123-456, I-test! test-990 650-555-1234"
}

https://www.elastic.co/guide/en/elasticsearch/reference/7.1/indices-analyze.html

3.2 Analyzer Composition and working mechanism

A word breaker is a component that deals with word segmentation , It's made up of three parts .Character Filters( For original text processing ),Tokenizer( Divide the words according to the rules ),Token Filter( Process the segmented words , A lowercase letter , Delete stopwords, Add synonyms )

3.3 ES Some built-in word breakers

• Standard Analyzer Default participator , Segmentation by words , Lowercase processing
• Simple Analyzer Cut by non letter , Lowercase processing
• Stop Analyzer
• Whitespace Analyzer
• Keyword Analyzer No participle , Treat input as output directly
• Patter Analyzer
• Language Provides 30 A word breaker for many common languages
• Customer Analyzer

ICU Analyzer
Need to follow plugin Elasticsearch-plugin install analysis-icu, Provides Unicode Support for , Better support for Asian Languages
IK Chinese word segmentation

4、Search API

#URI Query
GET kibana_sample_data_ecommerce/_search?q=customer_first_name:Eddie

#REQUEST Body
POST kibana_sample_data_ecommerce/_search
{
  "profile": true,  
  "_source":["order_date"],
  "query":{
    "match_all": {}
  }
}

# Script fields 
GET kibana_sample_data_ecommerce/_search
{
  "script_fields": {
    "new_field": {
      "script": {
        "lang": "painless",
        "source": "doc['order_date'].value+'hello'"
      }
    }
  },
  "query": {
    "match_all": {}
  }
}

https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-uri-request.html

https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-request-body.html

5、 Aggregate analysis

# Statistics by bucket according to destination 
GET kibana_sample_data_flights/_search
{
	"size": 0,
	"aggs":{
		"flight_dest":{
			"terms":{
				"field":"DestCountry"
			}
		}
	}
}



# View the statistics of flight destination , Increase the average , The highest and lowest price 
GET kibana_sample_data_flights/_search
{
	"size": 0,
	"aggs":{
		"flight_dest":{
			"terms":{
				"field":"DestCountry"
			},
			"aggs":{
				"avg_price":{
					"avg":{
						"field":"AvgTicketPrice"
					}
				},
				"max_price":{
					"max":{
						"field":"AvgTicketPrice"
					}
				},
				"min_price":{
					"min":{
						"field":"AvgTicketPrice"
					}
				}
			}
		}
	}
}



# Price statistics + Weather information 
GET kibana_sample_data_flights/_search
{
	"size": 0,
	"aggs":{
		"flight_dest":{
			"terms":{
				"field":"DestCountry"
			},
			"aggs":{
				"stats_price":{
					"stats":{
						"field":"AvgTicketPrice"
					}
				},
				"wather":{
				  "terms": {
				    "field": "DestWeather",
				    "size": 5
				  }
				}

			}
		}
	}
}

https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-aggregations.html