elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

时间:2022-07-05 04:13:23

一、分词器

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

1. 认识分词器

 1.1 Analyzer   分析器

在ES中一个Analyzer 由下面三种组件组合而成:

character filter :字符过滤器,对文本进行字符过滤处理,如处理文本中的html标签字符。处理完后再交给tokenizer进行分词。一个analyzer中可包含0个或多个字符过滤器,多个按配置顺序依次进行处理
tokenizer:分词器,对文本进行分词。一个analyzer必需且只可包含一个tokenizer
token filter:词项过滤器,对tokenizer分出的词进行过滤处理。如转小写、停用词处理、同义词处理。一个analyzer可包含0个或多个词项过滤器,按配置顺序进行过滤

1.2 如何测试分词器

POST _analyze
{
"analyzer": "whitespace",
"text": "The quick brown fox."
} POST _analyze
{
"tokenizer": "standard",
"filter": [ "lowercase", "asciifolding" ],
"text": "Is this déja vu?"
}

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

position:第几个词

offset:词的偏移位置

2. 内建的character filter

HTML Strip Character Filter
  html_strip :过滤html标签,解码HTML entities like &.
Mapping Character Filter
  mapping :用指定的字符串替换文本中的某字符串。
Pattern Replace Character Filter
  pattern_replace :进行正则表达式替换。

2.1 HTML Strip Character Filter 

POST _analyze
{
"tokenizer": "keyword",
"char_filter": [ "html_strip" ],
"text": "<p>I&apos;m so <b>happy</b>!</p>"
}

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

在索引中配置:

PUT my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "keyword",
"char_filter": ["my_char_filter"]
}
},
"char_filter": {
"my_char_filter": {
"type": "html_strip",
"escaped_tags": ["b"]
}
}

}
}
}

escaped_tags 用来指定例外的标签。 如果没有例外标签需配置,则不需要在此进行客户化定义,在上面的my_analyzer中直接使用 html_strip

测试:

POST my_index/_analyze
{
"analyzer": "my_analyzer",
"text": "<p>I&apos;m so <b>happy</b>!</p>"
}

2.2 Mapping character filter

官网链接:https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-mapping-charfilter.html

PUT my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "keyword",
"char_filter": [
"my_char_filter"
]
}
},
"char_filter": {
"my_char_filter": {
"type": "mapping",
"mappings": [
"٠ => 0",
"١ => 1",
"٢ => 2",
"٣ => 3",
"٤ => 4",
"٥ => 5",
"٦ => 6",
"٧ => 7",
"٨ => 8",
"٩ => 9"
]
}
}
}
}
}

测试

POST my_index/_analyze
{
"analyzer": "my_analyzer",
"text": "My license plate is ٢٥٠١٥"
}

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

2.3 Pattern Replace Character Filter

官网链接:https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-pattern-replace-charfilter.html

PUT my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"char_filter": [
"my_char_filter"
]
}
},
"char_filter": {
"my_char_filter": {
"type": "pattern_replace",
"pattern": "(\\d+)-(?=\\d)",
"replacement": "$1_"
}
}
}
}
}

测试

POST my_index/_analyze
{
"analyzer": "my_analyzer",
"text": "My credit card is 123-456-789"
}

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

3. 内建的Tokenizer

官网链接:https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-tokenizers.html

Standard Tokenizer
Letter Tokenizer
Lowercase Tokenizer
Whitespace Tokenizer
UAX URL Email Tokenizer
Classic Tokenizer
Thai Tokenizer
NGram Tokenizer
Edge NGram Tokenizer
Keyword Tokenizer
Pattern Tokenizer
Simple Pattern Tokenizer
Simple Pattern Split Tokenizer
Path Hierarchy Tokenizer

前面集成的中文分词器Ikanalyzer中提供的tokenizer:ik_smart 、 ik_max_word

 测试tokenizer

POST _analyze
{
"tokenizer": "standard",
"text": "张三说的确实在理"
} POST _analyze
{
"tokenizer": "ik_smart",
"text": "张三说的确实在理"
}

4.  内建的Token Filter

ES中内建了很多Token filter ,详细了解:https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-tokenizers.html

Lowercase Token Filter :lowercase 转小写
Stop Token Filter :stop 停用词过滤器
Synonym Token Filter: synonym 同义词过滤器

说明:中文分词器Ikanalyzer中自带有停用词过滤功能。

 4.1 Synonym Token Filter 同义词过滤器

PUT /test_index
{
"settings": {
"index" : {
"analysis" : {
"analyzer" : {
"my_ik_synonym" : {
"tokenizer" : "ik_smart",
"filter" : ["synonym"]
}
},
"filter" : {
"synonym" : {
"type" : "synonym",
<!-- synonyms_path:指定同义词文件(相对config的位置)-->
"synonyms_path" : "analysis/synonym.txt"
}
}
}
}
}
}

同义词定义格式

ES同义词格式支持 solr、 WordNet 两种格式。

在analysis/synonym.txt中用solr格式定义如下同义词

张三,李四
电饭煲,电饭锅 => 电饭煲
电脑 => 计算机,computer

注意:

文件一定要UTF-8编码

一行一类同义词,=> 表示标准化为

测试:通过例子的结果了解同义词的处理行为

POST test_index/_analyze
{
"analyzer": "my_ik_synonym",
"text": "张三说的确实在理"
} POST test_index/_analyze
{
"analyzer": "my_ik_synonym",
"text": "我想买个电饭锅和一个电脑"
}

5. 内建的Analyzer

官网链接:

https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html

Standard Analyzer
Simple Analyzer
Whitespace Analyzer
Stop Analyzer
Keyword Analyzer
Pattern Analyzer
Language Analyzers
Fingerprint Analyzer

集成的中文分词器Ikanalyzer中提供的Analyzer:ik_smart 、 ik_max_word

内建的和集成的analyzer可以直接使用。如果它们不能满足我们的需要,则我们可自己组合字符过滤器、分词器、词项过滤器来定义自定义的analyzer

5.1 自定义 Analyzer

配置参数:

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

PUT my_index8
{
"settings": {
"analysis": {
"analyzer": {
"my_ik_analyzer": {
"type": "custom",
"tokenizer": "ik_smart",
"char_filter": [
"html_strip"
],
"filter": [
"synonym"
]
}
},
"filter": {
"synonym": {
"type": "synonym",
"synonyms_path": "analysis/synonym.txt"
}
} } }}

 5.2 为字段指定分词器

PUT my_index8/_mapping/_doc
{
"properties": {
"title": {
"type": "text",
"analyzer": "my_ik_analyzer"
}
}
}

如果该字段的查询需要使用不同的analyzer

PUT my_index8/_mapping/_doc
{
"properties": {
"title": {
"type": "text",
"analyzer": "my_ik_analyzer",
"search_analyzer": "other_analyzer"
}
}
}

测试结果

PUT my_index8/_doc/1
{
"title": "张三说的确实在理"
} GET /my_index8/_search
{
"query": {
"term": {
"title": "张三"
}
}
}

 5.3 为索引定义个default分词器

PUT /my_index10
{
"settings": {
"analysis": {
"analyzer": {
"default": {
"tokenizer": "ik_smart",
"filter": [
"synonym"
]
}
},
"filter": {
"synonym": {
"type": "synonym",
"synonyms_path": "analysis/synonym.txt"
}
}
}
},
"mappings": {
"_doc": {
"properties": {
"title": {
"type": "text"
}
}
}
}
}

测试结果:

PUT my_index10/_doc/1
{
"title": "张三说的确实在理"
} GET /my_index10/_search
{
"query": {
"term": {
"title": "张三"
}
}
}

6. Analyzer的使用顺序

我们可以为每个查询、每个字段、每个索引指定分词器。

在索引阶段ES将按如下顺序来选用分词:

首先选用字段mapping定义中指定的analyzer
字段定义中没有指定analyzer,则选用 index settings中定义的名字为default 的analyzer。
如index setting中没有定义default分词器,则使用 standard analyzer.

查询阶段ES将按如下顺序来选用分词:

The analyzer defined in a full-text query.
The search_analyzer defined in the field mapping.
The analyzer defined in the field mapping.
An analyzer named default_search in the index settings.
An analyzer named default in the index settings.
The standard analyzer.

二、文档管理

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

1. 新建文档

指定文档id,新增/修改

PUT twitter/_doc/
{
"id": 1,
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}

新增,自动生成文档id

POST twitter/_doc/
{
"id": 1,
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}

返回结果说明:

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

2. 获取单个文档

HEAD twitter/_doc/

GET twitter/_doc/

不获取文档的source:

GET twitter/_doc/1?_source=false

获取文档的source:

GET twitter/_doc/1/_source

{
"_index": "twitter",
"_type": "_doc",
"_id": "1",
"_version": 2,
"found": true,
"_source": {
"id": 1,
"user": "kimchy",
"post_date": "2009-11-15T14:12:12",
"message": "trying out Elasticsearch"
}}

获取存储字段

PUT twitter11
{
"mappings": {
"_doc": {
"properties": {
"counter": {
"type": "integer",
"store": false
},
"tags": {
"type": "keyword",
"store": true
} } } }} PUT twitter11/_doc/1
{
"counter" : 1,
"tags" : ["red"]
} GET twitter11/_doc/1?stored_fields=tags,counter

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

3. 获取多个文档 _mget

方式1:

GET /_mget
{
"docs" : [
{
"_index" : "twitter",
"_type" : "_doc",
"_id" : "1"
},
{
"_index" : "twitter",
"_type" : "_doc",
"_id" : "2"
"stored_fields" : ["field3", "field4"]
}
]
}

方式2:

GET /twitter/_mget
{
"docs" : [
{
"_type" : "_doc",
"_id" : "1"
},
{
"_type" : "_doc",
"_id" : "2"
}
]
}

方式3:

GET /twitter/_doc/_mget
{
"docs" : [
{
"_id" : "1"
},
{
"_id" : "2"
}
]
}

方式4:

GET /twitter/_doc/_mget
{
"ids" : ["1", "2"]
}

4. 删除文档

指定文档id进行删除

DELETE twitter/_doc/1

用版本来控制删除

DELETE twitter/_doc/1?version=1

返回结果:

{
"_shards" : {
"total" : 2,
"failed" : 0,
"successful" : 2
},
"_index" : "twitter",
"_type" : "_doc",
"_id" : "1",
"_version" : 2,
"_primary_term": 1,
"_seq_no": 5,
"result": "deleted"
}

查询删除

POST twitter/_delete_by_query
{
"query": {
"match": {
"message": "some message"
}
}
}

当有文档有版本冲突时,不放弃删除操作(记录冲突的文档,继续删除其他复合查询的文档)

POST twitter/_doc/_delete_by_query?conflicts=proceed
{
"query": {
"match_all": {}
}
}

通过task api 来查看 查询删除任务

GET _tasks?detailed=true&actions=*/delete/byquery

查询具体任务的状态

GET /_tasks/taskId:1

取消任务

POST _tasks/task_id:1/_cancel

5. 更新文档

指定文档id进行修改

PUT twitter/_doc/
{
"id": 1,
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}

乐观锁并发更新控制

PUT twitter/_doc/1?version=1
{
"id": 1,
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}

返回结果

{
"_index": "twitter",
"_type": "_doc",
"_id": "1",
"_version": 3,
"result": "updated",
"_shards": {
"total": 3,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 3
}

6.Scripted update 通过脚本来更新文档

6.1 准备一个文档

PUT uptest/_doc/1
{
"counter" : 1,
"tags" : ["red"]
}

6.2、对文档1的counter + 4

POST uptest/_doc/1/_update
{
"script" : {
"source": "ctx._source.counter += params.count",
"lang": "painless",
"params" : {
"count" : 4
}
}
}

6.3、往数组中加入元素

POST uptest/_doc/1/_update
{
"script" : {
"source": "ctx._source.tags.add(params.tag)",
"lang": "painless",
"params" : {
"tag" : "blue"
}
}
}

脚本说明:painless是es内置的一种脚本语言,ctx执行上下文对象(通过它还可访问_index, _type, _id, _version, _routing and _now (the current timestamp) ),params是参数集合

 说明:脚本更新要求索引的_source 字段是启用的。更新执行流程:

a、获取到原文档
b、通过_source字段的原始数据,执行脚本修改。
c、删除原索引文档
d、索引修改后的文档
它只是降低了一些网络往返,并减少了get和索引之间版本冲突的可能性。

6.4、添加一个字段

POST uptest/_doc/1/_update
{
"script" : "ctx._source.new_field = 'value_of_new_field'"
}

6.5、移除一个字段

POST uptest/_doc/1/_update
{
"script" : "ctx._source.remove('new_field')"
}

6.6、判断删除或不做什么

POST uptest/_doc/1/_update
{
"script" : {
"source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'none' }",
"lang": "painless",
"params" : {
"tag" : "green"
}
}
}

6.7、合并传人的文档字段进行更新

POST uptest/_doc/1/_update
{
"doc" : {
"name" : "new_name"
}
}

6.8、再次执行7,更新内容相同,不需做什么

{
"_index": "uptest",
"_type": "_doc",
"_id": "1",
"_version": 4,
"result": "noop",
"_shards": {
"total": 0,
"successful": 0,
"failed": 0
}
}

6.9、设置不做noop检测

POST uptest/_doc/1/_update
{
"doc" : {
"name" : "new_name"
},
"detect_noop": false
}

什么是noop检测?

即已经执行过的脚本不再执行

6.10、upsert 操作:如果要更新的文档存在,则执行脚本进行更新,如不存在,则把 upsert中的内容作为一个新文档写入。

POST uptest/_doc/1/_update
{
"script" : {
"source": "ctx._source.counter += params.count",
"lang": "painless",
"params" : {
"count" : 4
}
},
"upsert" : {
"counter" : 1
}
}

7. 通过条件查询来更新文档

满足查询条件的才更新

POST twitter/_update_by_query
{
"script": {
"source": "ctx._source.likes++",
"lang": "painless"
},
"query": {
"term": {
"user": "kimchy"
}
}
}

8. 批量操作

批量操作API /_bulk 让我们可以在一次调用中执行多个索引、删除操作。这可以大大提高索引数据的速度。批量操作内容体需按如下以新行分割的json结构格式给出:

语法:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

说明:

action_and_meta_data: action可以是 index, create, delete and update ,meta_data 指: _index ,_type,_id 请求端点可以是: /_bulk, /{index}/_bulk, {index}/{type}/_bulk

示例:

POST _bulk
{ "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "_doc", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "_doc", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "_doc", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }

8.1 curl + json 文件 批量索引多个文档

注意:accounts.json要放在执行curl命令的同等级目录下,后续学习的测试数据基本都使用这份银行的数据了

curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"

accounts.json:

{"index":{"_id":"1"}}
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"amberduke@pyrami.com","city":"Brogan","state":"IL"}
{"index":{"_id":"6"}}
{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"hattiebond@netagy.com","city":"Dante","state":"TN"}
{"index":{"_id":"13"}}
{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","email":"nanettebates@quility.com","city":"Nogal","state":"VA"}
{"index":{"_id":"18"}}
{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","employer":"Boink","email":"daleadams@boink.com","city":"Orick","state":"MD"}
{"index":{"_id":"20"}}
{"account_number":20,"balance":16418,"firstname":"Elinor","lastname":"Ratliff","age":36,"gender":"M","address":"282 Kings Place","employer":"Scentric","email":"elinorratliff@scentric.com","city":"Ribera","state":"WA"}
{"index":{"_id":"25"}}
{"account_number":25,"balance":40540,"firstname":"Virginia","lastname":"Ayala","age":39,"gender":"F","address":"171 Putnam Avenue","employer":"Filodyne","email":"virginiaayala@filodyne.com","city":"Nicholson","state":"PA"}
{"index":{"_id":"32"}}
{"account_number":32,"balance":48086,"firstname":"Dillard","lastname":"Mcpherson","age":34,"gender":"F","address":"702 Quentin Street","employer":"Quailcom","email":"dillardmcpherson@quailcom.com","city":"Veguita","state":"IN"}
{"index":{"_id":"37"}}
{"account_number":37,"balance":18612,"firstname":"Mcgee","lastname":"Mooney","age":39,"gender":"M","address":"826 Fillmore Place","employer":"Reversus","email":"mcgeemooney@reversus.com","city":"Tooleville","state":"OK"}
{"index":{"_id":"44"}}
{"account_number":44,"balance":34487,"firstname":"Aurelia","lastname":"Harding","age":37,"gender":"M","address":"502 Baycliff Terrace","employer":"Orbalix","email":"aureliaharding@orbalix.com","city":"Yardville","state":"DE"}
{"index":{"_id":"49"}}
{"account_number":49,"balance":29104,"firstname":"Fulton","lastname":"Holt","age":23,"gender":"F","address":"451 Humboldt Street","employer":"Anocha","email":"fultonholt@anocha.com","city":"Sunriver","state":"RI"}
{"index":{"_id":"51"}}
{"account_number":51,"balance":14097,"firstname":"Burton","lastname":"Meyers","age":31,"gender":"F","address":"334 River Street","employer":"Bezal","email":"burtonmeyers@bezal.com","city":"Jacksonburg","state":"MO"}
{"index":{"_id":"56"}}
{"account_number":56,"balance":14992,"firstname":"Josie","lastname":"Nelson","age":32,"gender":"M","address":"857 Tabor Court","employer":"Emtrac","email":"josienelson@emtrac.com","city":"Sunnyside","state":"UT"}
{"index":{"_id":"63"}}
{"account_number":63,"balance":6077,"firstname":"Hughes","lastname":"Owens","age":30,"gender":"F","address":"510 Sedgwick Street","employer":"Valpreal","email":"hughesowens@valpreal.com","city":"Guilford","state":"KS"}
{"index":{"_id":"68"}}
{"account_number":68,"balance":44214,"firstname":"Hall","lastname":"Key","age":25,"gender":"F","address":"927 Bay Parkway","employer":"Eventex","email":"hallkey@eventex.com","city":"Shawmut","state":"CA"}
{"index":{"_id":"70"}}
{"account_number":70,"balance":38172,"firstname":"Deidre","lastname":"Thompson","age":33,"gender":"F","address":"685 School Lane","employer":"Netplode","email":"deidrethompson@netplode.com","city":"Chestnut","state":"GA"}
{"index":{"_id":"75"}}
{"account_number":75,"balance":40500,"firstname":"Sandoval","lastname":"Kramer","age":22,"gender":"F","address":"166 Irvington Place","employer":"Overfork","email":"sandovalkramer@overfork.com","city":"Limestone","state":"NH"}
{"index":{"_id":"82"}}
{"account_number":82,"balance":41412,"firstname":"Concetta","lastname":"Barnes","age":39,"gender":"F","address":"195 Bayview Place","employer":"Fitcore","email":"concettabarnes@fitcore.com","city":"Summerfield","state":"NC"}
{"index":{"_id":"87"}}
{"account_number":87,"balance":1133,"firstname":"Hewitt","lastname":"Kidd","age":22,"gender":"M","address":"446 Halleck Street","employer":"Isologics","email":"hewittkidd@isologics.com","city":"Coalmont","state":"ME"}
{"index":{"_id":"94"}}
{"account_number":94,"balance":41060,"firstname":"Brittany","lastname":"Cabrera","age":30,"gender":"F","address":"183 Kathleen Court","employer":"Mixers","email":"brittanycabrera@mixers.com","city":"Cornucopia","state":"AZ"}
{"index":{"_id":"99"}}
{"account_number":99,"balance":47159,"firstname":"Ratliff","lastname":"Heath","age":39,"gender":"F","address":"806 Rockwell Place","employer":"Zappix","email":"ratliffheath@zappix.com","city":"Shaft","state":"ND"}
{"index":{"_id":"102"}}
{"account_number":102,"balance":29712,"firstname":"Dena","lastname":"Olson","age":27,"gender":"F","address":"759 Newkirk Avenue","employer":"Hinway","email":"denaolson@hinway.com","city":"Choctaw","state":"NJ"}
{"index":{"_id":"107"}}
{"account_number":107,"balance":48844,"firstname":"Randi","lastname":"Rich","age":28,"gender":"M","address":"694 Jefferson Street","employer":"Netplax","email":"randirich@netplax.com","city":"Bellfountain","state":"SC"}
{"index":{"_id":"114"}}
{"account_number":114,"balance":43045,"firstname":"Josephine","lastname":"Joseph","age":31,"gender":"F","address":"451 Oriental Court","employer":"Turnabout","email":"josephinejoseph@turnabout.com","city":"Sedley","state":"AL"}
{"index":{"_id":"119"}}
{"account_number":119,"balance":49222,"firstname":"Laverne","lastname":"Johnson","age":28,"gender":"F","address":"302 Howard Place","employer":"Senmei","email":"lavernejohnson@senmei.com","city":"Herlong","state":"DC"}
{"index":{"_id":"121"}}
{"account_number":121,"balance":19594,"firstname":"Acevedo","lastname":"Dorsey","age":32,"gender":"M","address":"479 Nova Court","employer":"Netropic","email":"acevedodorsey@netropic.com","city":"Islandia","state":"CT"}
{"index":{"_id":"126"}}
{"account_number":126,"balance":3607,"firstname":"Effie","lastname":"Gates","age":39,"gender":"F","address":"620 National Drive","employer":"Digitalus","email":"effiegates@digitalus.com","city":"Blodgett","state":"MD"}
{"index":{"_id":"133"}}
{"account_number":133,"balance":26135,"firstname":"Deena","lastname":"Richmond","age":36,"gender":"F","address":"646 Underhill Avenue","employer":"Sunclipse","email":"deenarichmond@sunclipse.com","city":"Austinburg","state":"SC"}
{"index":{"_id":"138"}}
{"account_number":138,"balance":9006,"firstname":"Daniel","lastname":"Arnold","age":39,"gender":"F","address":"422 Malbone Street","employer":"Ecstasia","email":"danielarnold@ecstasia.com","city":"Gardiner","state":"MO"}
{"index":{"_id":"140"}}
{"account_number":140,"balance":26696,"firstname":"Cotton","lastname":"Christensen","age":32,"gender":"M","address":"878 Schermerhorn Street","employer":"Prowaste","email":"cottonchristensen@prowaste.com","city":"Mayfair","state":"LA"}
{"index":{"_id":"145"}}
{"account_number":145,"balance":47406,"firstname":"Rowena","lastname":"Wilkinson","age":32,"gender":"M","address":"891 Elton Street","employer":"Asimiline","email":"rowenawilkinson@asimiline.com","city":"Ripley","state":"NH"}
{"index":{"_id":"152"}}
{"account_number":152,"balance":8088,"firstname":"Wolfe","lastname":"Rocha","age":21,"gender":"M","address":"457 Guernsey Street","employer":"Hivedom","email":"wolferocha@hivedom.com","city":"Adelino","state":"MS"}
{"index":{"_id":"157"}}
{"account_number":157,"balance":39868,"firstname":"Claudia","lastname":"Terry","age":20,"gender":"F","address":"132 Gunnison Court","employer":"Lumbrex","email":"claudiaterry@lumbrex.com","city":"Castleton","state":"MD"}
{"index":{"_id":"164"}}
{"account_number":164,"balance":9101,"firstname":"Cummings","lastname":"Little","age":26,"gender":"F","address":"308 Schaefer Street","employer":"Comtrak","email":"cummingslittle@comtrak.com","city":"Chaparrito","state":"WI"}
{"index":{"_id":"169"}}
{"account_number":169,"balance":45953,"firstname":"Hollie","lastname":"Osborn","age":34,"gender":"M","address":"671 Seaview Court","employer":"Musaphics","email":"hollieosborn@musaphics.com","city":"Hanover","state":"GA"}
{"index":{"_id":"171"}}
{"account_number":171,"balance":7091,"firstname":"Nelda","lastname":"Hopper","age":39,"gender":"M","address":"742 Prospect Place","employer":"Equicom","email":"neldahopper@equicom.com","city":"Finderne","state":"SC"}
{"index":{"_id":"176"}}
{"account_number":176,"balance":18607,"firstname":"Kemp","lastname":"Walters","age":28,"gender":"F","address":"906 Howard Avenue","employer":"Eyewax","email":"kempwalters@eyewax.com","city":"Why","state":"KY"}
{"index":{"_id":"183"}}
{"account_number":183,"balance":14223,"firstname":"Hudson","lastname":"English","age":26,"gender":"F","address":"823 Herkimer Place","employer":"Xinware","email":"hudsonenglish@xinware.com","city":"Robbins","state":"ND"}
{"index":{"_id":"188"}}
{"account_number":188,"balance":41504,"firstname":"Tia","lastname":"Miranda","age":24,"gender":"F","address":"583 Ainslie Street","employer":"Jasper","email":"tiamiranda@jasper.com","city":"Summerset","state":"UT"}
{"index":{"_id":"190"}}
{"account_number":190,"balance":3150,"firstname":"Blake","lastname":"Davidson","age":30,"gender":"F","address":"636 Diamond Street","employer":"Quantasis","email":"blakedavidson@quantasis.com","city":"Crumpler","state":"KY"}
{"index":{"_id":"195"}}
{"account_number":195,"balance":5025,"firstname":"Kaye","lastname":"Gibson","age":31,"gender":"M","address":"955 Hopkins Street","employer":"Zork","email":"kayegibson@zork.com","city":"Ola","state":"WY"}
{"index":{"_id":"203"}}
{"account_number":203,"balance":21890,"firstname":"Eve","lastname":"Wyatt","age":33,"gender":"M","address":"435 Furman Street","employer":"Assitia","email":"evewyatt@assitia.com","city":"Jamestown","state":"MN"}
{"index":{"_id":"208"}}
{"account_number":208,"balance":40760,"firstname":"Garcia","lastname":"Hess","age":26,"gender":"F","address":"810 Nostrand Avenue","employer":"Quiltigen","email":"garciahess@quiltigen.com","city":"*trails","state":"GA"}
{"index":{"_id":"210"}}
{"account_number":210,"balance":33946,"firstname":"Cherry","lastname":"Carey","age":24,"gender":"M","address":"539 Tiffany Place","employer":"Martgo","email":"cherrycarey@martgo.com","city":"Fairacres","state":"AK"}
{"index":{"_id":"215"}}
{"account_number":215,"balance":37427,"firstname":"Copeland","lastname":"Solomon","age":20,"gender":"M","address":"741 McDonald Avenue","employer":"Recognia","email":"copelandsolomon@recognia.com","city":"Edmund","state":"ME"}
{"index":{"_id":"222"}}
{"account_number":222,"balance":14764,"firstname":"Rachelle","lastname":"Rice","age":36,"gender":"M","address":"333 Narrows Avenue","employer":"Enaut","email":"rachellerice@enaut.com","city":"Wright","state":"AZ"}
{"index":{"_id":"227"}}
{"account_number":227,"balance":19780,"firstname":"Coleman","lastname":"Berg","age":22,"gender":"M","address":"776 Little Street","employer":"Exoteric","email":"colemanberg@exoteric.com","city":"Eagleville","state":"WV"}
{"index":{"_id":"234"}}
{"account_number":234,"balance":44207,"firstname":"Betty","lastname":"Hall","age":37,"gender":"F","address":"709 Garfield Place","employer":"Miraclis","email":"bettyhall@miraclis.com","city":"Bendon","state":"NY"}
{"index":{"_id":"239"}}
{"account_number":239,"balance":25719,"firstname":"Chang","lastname":"Boyer","age":36,"gender":"M","address":"895 Brigham Street","employer":"Qaboos","email":"changboyer@qaboos.com","city":"Belgreen","state":"NH"}
{"index":{"_id":"241"}}
{"account_number":241,"balance":25379,"firstname":"Schroeder","lastname":"Harrington","age":26,"gender":"M","address":"610 Tapscott Avenue","employer":"Otherway","email":"schroederharrington@otherway.com","city":"Ebro","state":"TX"}
{"index":{"_id":"246"}}
{"account_number":246,"balance":28405,"firstname":"Katheryn","lastname":"Foster","age":21,"gender":"F","address":"259 Kane Street","employer":"Quantalia","email":"katherynfoster@quantalia.com","city":"Bath","state":"TX"}
{"index":{"_id":"253"}}
{"account_number":253,"balance":20240,"firstname":"Melissa","lastname":"Gould","age":31,"gender":"M","address":"440 Fuller Place","employer":"Buzzopia","email":"melissagould@buzzopia.com","city":"Lumberton","state":"MD"}
{"index":{"_id":"258"}}
{"account_number":258,"balance":5712,"firstname":"Lindsey","lastname":"Hawkins","age":37,"gender":"M","address":"706 Frost Street","employer":"Enormo","email":"lindseyhawkins@enormo.com","city":"Gardners","state":"AK"}
{"index":{"_id":"260"}}
{"account_number":260,"balance":2726,"firstname":"Kari","lastname":"Skinner","age":30,"gender":"F","address":"735 Losee Terrace","employer":"Singavera","email":"kariskinner@singavera.com","city":"Rushford","state":"WV"}
{"index":{"_id":"265"}}
{"account_number":265,"balance":46910,"firstname":"Marion","lastname":"Schneider","age":26,"gender":"F","address":"574 Everett Avenue","employer":"Evidends","email":"marionschneider@evidends.com","city":"Maplewood","state":"WY"}
{"index":{"_id":"272"}}
{"account_number":272,"balance":19253,"firstname":"Lilly","lastname":"Morgan","age":25,"gender":"F","address":"689 Fleet Street","employer":"Biolive","email":"lillymorgan@biolive.com","city":"Sunbury","state":"OH"}
{"index":{"_id":"277"}}
{"account_number":277,"balance":29564,"firstname":"Romero","lastname":"Lott","age":31,"gender":"M","address":"456 Danforth Street","employer":"Plasto","email":"romerolott@plasto.com","city":"Vincent","state":"VT"}
{"index":{"_id":"284"}}
{"account_number":284,"balance":22806,"firstname":"Randolph","lastname":"Banks","age":29,"gender":"M","address":"875 Hamilton Avenue","employer":"Caxt","email":"randolphbanks@caxt.com","city":"Crawfordsville","state":"WA"}
{"index":{"_id":"289"}}
{"account_number":289,"balance":7798,"firstname":"Blair","lastname":"Church","age":29,"gender":"M","address":"370 Sutton Street","employer":"Cubix","email":"blairchurch@cubix.com","city":"Nile","state":"NH"}
{"index":{"_id":"291"}}
{"account_number":291,"balance":19955,"firstname":"Lynn","lastname":"Pollard","age":40,"gender":"F","address":"685 Pierrepont Street","employer":"Slambda","email":"lynnpollard@slambda.com","city":"Mappsville","state":"ID"}
{"index":{"_id":"296"}}
{"account_number":296,"balance":24606,"firstname":"Rosa","lastname":"Oliver","age":34,"gender":"M","address":"168 Woodbine Street","employer":"Idetica","email":"rosaoliver@idetica.com","city":"Robinson","state":"WY"}
{"index":{"_id":"304"}}
{"account_number":304,"balance":28647,"firstname":"Palmer","lastname":"Clark","age":35,"gender":"M","address":"866 Boulevard Court","employer":"Maximind","email":"palmerclark@maximind.com","city":"Avalon","state":"NH"}
{"index":{"_id":"309"}}
{"account_number":309,"balance":3830,"firstname":"Rosemarie","lastname":"Nieves","age":30,"gender":"M","address":"206 Alice Court","employer":"Zounds","email":"rosemarienieves@zounds.com","city":"Ferney","state":"AR"}
{"index":{"_id":"311"}}
{"account_number":311,"balance":13388,"firstname":"Vinson","lastname":"Ballard","age":23,"gender":"F","address":"960 Glendale Court","employer":"Gynk","email":"vinsonballard@gynk.com","city":"Fairforest","state":"WY"}
{"index":{"_id":"316"}}
{"account_number":316,"balance":8214,"firstname":"Anita","lastname":"Ewing","age":32,"gender":"M","address":"396 Lombardy Street","employer":"Panzent","email":"anitaewing@panzent.com","city":"Neahkahnie","state":"WY"}
{"index":{"_id":"323"}}
{"account_number":323,"balance":42230,"firstname":"Chelsea","lastname":"Gamble","age":34,"gender":"F","address":"356 Dare Court","employer":"Isosphere","email":"chelseagamble@isosphere.com","city":"Dundee","state":"MD"}
{"index":{"_id":"328"}}
{"account_number":328,"balance":12523,"firstname":"Good","lastname":"Campbell","age":27,"gender":"F","address":"438 Hicks Street","employer":"Gracker","email":"goodcampbell@gracker.com","city":"Marion","state":"CA"}
{"index":{"_id":"330"}}
{"account_number":330,"balance":41620,"firstname":"Yvette","lastname":"Browning","age":34,"gender":"F","address":"431 Beekman Place","employer":"Marketoid","email":"yvettebrowning@marketoid.com","city":"Talpa","state":"CO"}
{"index":{"_id":"335"}}
{"account_number":335,"balance":35433,"firstname":"Vera","lastname":"Hansen","age":24,"gender":"M","address":"252 Bushwick Avenue","employer":"Zanilla","email":"verahansen@zanilla.com","city":"Manila","state":"TN"}
{"index":{"_id":"342"}}
{"account_number":342,"balance":33670,"firstname":"Vivian","lastname":"Wells","age":36,"gender":"M","address":"570 Cobek Court","employer":"Nutralab","email":"vivianwells@nutralab.com","city":"Fontanelle","state":"OK"}
{"index":{"_id":"347"}}
{"account_number":347,"balance":36038,"firstname":"Gould","lastname":"Carson","age":24,"gender":"F","address":"784 Pulaski Street","employer":"Mobildata","email":"gouldcarson@mobildata.com","city":"Goochland","state":"MI"}
{"index":{"_id":"354"}}
{"account_number":354,"balance":21294,"firstname":"Kidd","lastname":"Mclean","age":22,"gender":"M","address":"691 Saratoga Avenue","employer":"Ronbert","email":"kiddmclean@ronbert.com","city":"Tioga","state":"ME"}
{"index":{"_id":"359"}}
{"account_number":359,"balance":29927,"firstname":"Vanessa","lastname":"Harvey","age":28,"gender":"F","address":"679 Rutledge Street","employer":"Zentime","email":"vanessaharvey@zentime.com","city":"Williston","state":"IL"}
{"index":{"_id":"361"}}
{"account_number":361,"balance":23659,"firstname":"Noreen","lastname":"Shelton","age":36,"gender":"M","address":"702 Tillary Street","employer":"Medmex","email":"noreenshelton@medmex.com","city":"Derwood","state":"NH"}
{"index":{"_id":"366"}}
{"account_number":366,"balance":42368,"firstname":"Lydia","lastname":"Cooke","age":31,"gender":"M","address":"470 Coleman Street","employer":"Comstar","email":"lydiacooke@comstar.com","city":"Datil","state":"TN"}
{"index":{"_id":"373"}}
{"account_number":373,"balance":9671,"firstname":"Simpson","lastname":"Carpenter","age":21,"gender":"M","address":"837 Horace Court","employer":"Snips","email":"simpsoncarpenter@snips.com","city":"Tolu","state":"MA"}
{"index":{"_id":"378"}}
{"account_number":378,"balance":27100,"firstname":"Watson","lastname":"Simpson","age":36,"gender":"F","address":"644 Thomas Street","employer":"Wrapture","email":"watsonsimpson@wrapture.com","city":"Keller","state":"TX"}
{"index":{"_id":"380"}}
{"account_number":380,"balance":35628,"firstname":"Fernandez","lastname":"Reid","age":33,"gender":"F","address":"154 Melba Court","employer":"Cosmosis","email":"fernandezreid@cosmosis.com","city":"Boyd","state":"NE"}
{"index":{"_id":"385"}}
{"account_number":385,"balance":11022,"firstname":"Rosalinda","lastname":"Valencia","age":22,"gender":"M","address":"933 Lloyd Street","employer":"Zoarere","email":"rosalindavalencia@zoarere.com","city":"Waverly","state":"GA"}
{"index":{"_id":"392"}}
{"account_number":392,"balance":31613,"firstname":"Dotson","lastname":"Dean","age":35,"gender":"M","address":"136 Ford Street","employer":"Petigems","email":"dotsondean@petigems.com","city":"Chical","state":"SD"}
{"index":{"_id":"397"}}
{"account_number":397,"balance":37418,"firstname":"Leonard","lastname":"Gray","age":36,"gender":"F","address":"840 Morgan Avenue","employer":"Recritube","email":"leonardgray@recritube.com","city":"Edenburg","state":"AL"}
{"index":{"_id":"400"}}
{"account_number":400,"balance":20685,"firstname":"Kane","lastname":"King","age":21,"gender":"F","address":"405 Cornelia Street","employer":"Tri@Tribalog","email":"kaneking@tri@tribalog.com","city":"Gulf","state":"VT"}
{"index":{"_id":"405"}}
{"account_number":405,"balance":5679,"firstname":"Strickland","lastname":"Fuller","age":26,"gender":"M","address":"990 Concord Street","employer":"Digique","email":"stricklandfuller@digique.com","city":"Southmont","state":"NV"}
{"index":{"_id":"412"}}
{"account_number":412,"balance":27436,"firstname":"Ilene","lastname":"Abbott","age":26,"gender":"M","address":"846 Vine Street","employer":"Typhonica","email":"ileneabbott@typhonica.com","city":"Cedarville","state":"VT"}
{"index":{"_id":"417"}}
{"account_number":417,"balance":1788,"firstname":"Wheeler","lastname":"Ayers","age":35,"gender":"F","address":"677 Hope Street","employer":"Fortean","email":"wheelerayers@fortean.com","city":"Ironton","state":"PA"}
{"index":{"_id":"424"}}
{"account_number":424,"balance":36818,"firstname":"Tracie","lastname":"Gregory","age":34,"gender":"M","address":"112 Hunterfly Place","employer":"Comstruct","email":"traciegregory@comstruct.com","city":"Onton","state":"TN"}
{"index":{"_id":"429"}}
{"account_number":429,"balance":46970,"firstname":"Cantu","lastname":"Lindsey","age":31,"gender":"M","address":"404 Willoughby Avenue","employer":"Inquala","email":"cantulindsey@inquala.com","city":"Cowiche","state":"IA"}
{"index":{"_id":"431"}}
{"account_number":431,"balance":13136,"firstname":"Laurie","lastname":"Shaw","age":26,"gender":"F","address":"263 Aviation Road","employer":"Zillanet","email":"laurieshaw@zillanet.com","city":"Harmon","state":"WV"}
{"index":{"_id":"436"}}
{"account_number":436,"balance":27585,"firstname":"Alexander","lastname":"Sargent","age":23,"gender":"M","address":"363 Albemarle Road","employer":"Fangold","email":"alexandersargent@fangold.com","city":"Calpine","state":"OR"}
{"index":{"_id":"443"}}
{"account_number":443,"balance":7588,"firstname":"Huff","lastname":"Thomas","age":23,"gender":"M","address":"538 Erskine Loop","employer":"Accufarm","email":"huffthomas@accufarm.com","city":"Corinne","state":"AL"}
{"index":{"_id":"448"}}
{"account_number":448,"balance":22776,"firstname":"Adriana","lastname":"Mcfadden","age":35,"gender":"F","address":"984 Woodside Avenue","employer":"Telequiet","email":"adrianamcfadden@telequiet.com","city":"Darrtown","state":"WI"}
{"index":{"_id":"450"}}
{"account_number":450,"balance":2643,"firstname":"Bradford","lastname":"Nielsen","age":25,"gender":"M","address":"487 Keen Court","employer":"Exovent","email":"bradfordnielsen@exovent.com","city":"Hamilton","state":"DE"}
{"index":{"_id":"455"}}
{"account_number":455,"balance":39556,"firstname":"Lynn","lastname":"Tran","age":36,"gender":"M","address":"741 Richmond Street","employer":"Optyk","email":"lynntran@optyk.com","city":"Clinton","state":"WV"}
{"index":{"_id":"462"}}
{"account_number":462,"balance":10871,"firstname":"Calderon","lastname":"Day","age":27,"gender":"M","address":"810 Milford Street","employer":"Cofine","email":"calderonday@cofine.com","city":"Kula","state":"OK"}
{"index":{"_id":"467"}}
{"account_number":467,"balance":6312,"firstname":"Angelica","lastname":"May","age":32,"gender":"F","address":"384 Karweg Place","employer":"Keeg","email":"angelicamay@keeg.com","city":"Tetherow","state":"IA"}
{"index":{"_id":"474"}}
{"account_number":474,"balance":35896,"firstname":"Obrien","lastname":"Walton","age":40,"gender":"F","address":"192 Ide Court","employer":"Suremax","email":"obrienwalton@suremax.com","city":"Crucible","state":"UT"}
{"index":{"_id":"479"}}
{"account_number":479,"balance":31865,"firstname":"Cameron","lastname":"Ross","age":40,"gender":"M","address":"904 Bouck Court","employer":"Telpod","email":"cameronross@telpod.com","city":"Nord","state":"MO"}
{"index":{"_id":"481"}}
{"account_number":481,"balance":20024,"firstname":"Lina","lastname":"Stanley","age":33,"gender":"M","address":"361 Hanover Place","employer":"Strozen","email":"linastanley@strozen.com","city":"Wyoming","state":"NC"}
{"index":{"_id":"486"}}
{"account_number":486,"balance":35902,"firstname":"Dixie","lastname":"Fuentes","age":22,"gender":"F","address":"991 Applegate Court","employer":"Portico","email":"dixiefuentes@portico.com","city":"Salix","state":"VA"}
{"index":{"_id":"493"}}
{"account_number":493,"balance":5871,"firstname":"Campbell","lastname":"Best","age":24,"gender":"M","address":"297 Friel Place","employer":"Fanfare","email":"campbellbest@fanfare.com","city":"Kidder","state":"GA"}
{"index":{"_id":"498"}}
{"account_number":498,"balance":10516,"firstname":"Stella","lastname":"Hinton","age":39,"gender":"F","address":"649 Columbia Place","employer":"Flyboyz","email":"stellahinton@flyboyz.com","city":"Crenshaw","state":"SC"}

9. reindex 重索引

Reindex API /_reindex 让我们可以将一个索引中的数据重索引到另一个索引中(拷贝),要求源索引的_source 是开启的。目标索引的setting 、mapping 信息与源索引无关。

什么时候需要重索引?

即当需要做数据的拷贝的时候

POST _reindex
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter"
}
}

重索引要考虑的一个问题:目标索引中存在源索引中的数据,这些数据的version如何处理。

1、如果没有指定version_type 或指定为 internal,则会是采用目标索引中的版本,重索引过程中,执行的就是新增、更新操作。

POST _reindex
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter",
"version_type": "internal"
}
}

2、如果想使用源索引中的版本来进行版本控制更新,则设置 version_type 为extenal。重索引操作将写入不存在的,更新旧版本的数据。

POST _reindex
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter",
"version_type": "external"
}
}

如果你只想从源索引中复制目标索引中不存在的文档数据,可以指定 op_type 为 create 。此时存在的文档将触发 版本冲突(会导致放弃操作),可设置“conflicts”: “proceed“,跳过继续

POST _reindex
{
"conflicts": "proceed",
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter",
"op_type": "create"
}
}

你也可以只索引源索引的一部分数据,通过 type 或 查询来指定你需要的数据

POST _reindex
{
"source": {
"index": "twitter",
"type": "_doc",
"query": {
"term": {
"user": "kimchy"
}
}
},
"dest": {
"index": "new_twitter"
}
}

可以从多个源获取数据

POST _reindex
{
"source": {
"index": ["twitter", "blog"],
"type": ["_doc", "post"]
},
"dest": {
"index": "all_together"
}
}

可以限定文档数量

POST _reindex
{
"size": 10000,
"source": {
"index": "twitter",
"sort": { "date": "desc" }
},
"dest": {
"index": "new_twitter"
}
}

可以选择复制源文档的哪些字段

POST _reindex
{
"source": {
"index": "twitter",
"_source": ["user", "_doc"]
},
"dest": {
"index": "new_twitter"
}
}

可以用script来改变文档

POST _reindex
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter",
"version_type": "external"
},
"script": {
"source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
"lang": "painless"
}

}

可以指定路由值把文档放到哪个分片上

POST _reindex
{
"source": {
"index": "source",
"query": {
"match": {
"company": "cat"
}
}
},
"dest": {
"index": "dest",
"routing": "=cat"
}
}

从远程源复制

POST _reindex
{
"source": {
"remote": {
"host": "http://otherhost:9200",
"username": "user",
"password": "pass"
},
"index": "source",
"query": {
"match": {
"test": "data"
}
}

},
"dest": {
"index": "dest"
}
}

通过_task 来查询执行状态

GET _tasks?detailed=true&actions=*reindex

10. refresh

对于索引、更新、删除操作如果想操作完后立马重刷新可见,可带上refresh参数

PUT /test/_doc/1?refresh
{"test": "test"}
PUT /test/_doc/2?refresh=true
{"test": "test"}

refresh 可选值说明

未给值或=true,则立马会重刷新读索引。
=false ,相当于没带refresh 参数,遵循内部的定时刷新。
=wait_for ,登记等待刷新,当登记的请求数达到index.max_refresh_listeners 参数设定的值时(defaults to 1000),将触发重刷新。

三、路由详解

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

1. 集群组成

第一个节点启动

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

说明:首先启动的一定是主节点,主节点存储的是集群的元数据信息

Node2启动

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

说明:

Node2节点启动之前会配置集群的名称Cluster-name:ess,然后配置可以作为主节点的ip地址信息discovery.zen.ping.unicast.hosts: [“10.0.1.11",“10.0.1.12"],配置自己的ip地址networ.host: 10.0.1.12;

Node2启动的过程中会去找到主节点Node1告诉Node1我要加入到集群里面了,主节点Node1接收到请求以后看Node2是否满足加入集群的条件,如果满足就把node2的ip地址加入的元信息里面,然后广播给集群中的其他节点有

新节点加入,并把最新的元信息发送给其他的节点去更新

Node3..NodeN加入

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

说明:集群中的所有节点的元信息都是和主节点一致的,因为一旦有新的节点加入进来,主节点会通知其他的节点同步元信息

2. 在集群中创建索引的流程

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

3. 有索引的集群

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

4. 集群有节点出现故障,如主节点挂了,会重新选择主节点

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

5. 在集群中索引文档

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

索引文档的步骤:
1、node2计算文档的路由值得到文档存放的分片(假定路由选定的是分片0)。
2、将文档转发给分片0(P0)的主分片节点 node1。
3、node1索引文档,同步给副本(R0)节点node3索引文档。
4、node1向node2反馈结果
5、node2作出响应

6. 文档是如何路由的

文档该存到哪个分片上?
决定文档存放到哪个分片上就是文档路由。ES中通过下面的计算得到每个文档的存放分片:

shard = hash(routing) % number_of_primary_shards

参数说明:

routing 是用来进行hash计算的路由值,默认是使用文档id值。我们可以在索引文档时通过routing参数指定别的路由值

number_of_primary_shards:创建索引时指定的主分片数

POST twitter/_doc?routing=kimchy
{
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}

在索引、删除、更新、查询中都可以使用routing参数(可多值)指定操作的分片。

创建索引时强制要求给定路由值:

PUT my_index2
{
"mappings": {
"_doc": {
"_routing": {
"required": true
}
}
}
}

7. 在集群中进行搜索

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))

搜索的步骤:如要搜索 索引 s0
1、node2解析查询。
2、node2将查询发给索引s0的分片/副本(R1,R2,R0)节点
3、各节点执行查询,将结果发给Node2
4、Node2合并结果,作出响应。

8. Master节点的工作是什么?

1. 存储集群的元信息,如集群名称、集群中的节点

2. 转发创建索引和索引文档的请求

3. 和其他的节点进行通信,告诉其他节点有新的节点加入等

elasticsearch系列三:索引详解(分词器、文档管理、路由详解(集群))的更多相关文章

  1. ElasticSearch(三):通分词器(Analyzer)进行分词(Analysis)

    ElasticSearch(三):通过分词器(Analyzer)进行分词(Analysis) 学习课程链接<Elasticsearch核心技术与实战> Analysis与Analyzer ...

  2. elasticsearch最全详细使用教程:入门、索引管理、映射详解、索引别名、分词器、文档管理、路由、搜索详解

    一.快速入门1. 查看集群的健康状况http://localhost:9200/_cat http://localhost:9200/_cat/health?v 说明:v是用来要求在结果中返回表头 状 ...

  3. Linux下&comma;非Docker启动Elasticsearch 6&period;3&period;0&comma;安装ik分词器插件&comma;以及使用Kibana测试Elasticsearch&comma;

    Linux下,非Docker启动Elasticsearch 6.3.0 查看java版本,需要1.8版本 java -version yum -y install java 创建用户,因为elasti ...

  4. Elasticsearch从入门到放弃:分词器初印象

    Elasticsearch 系列回来了,先给因为这个系列关注我的同学说声抱歉,拖了这么久才回来,这个系列虽然叫「Elasticsearch 从入门到放弃」,但只有三篇就放弃还是有点过分的,所以还是回来 ...

  5. Elasticsearch之几个重要的分词器

    前提 什么是倒排索引? Elasticsearch之分词器的作用 Elasticsearch之分词器的工作流程 Elasticsearch之停用词 Elasticsearch之中文分词器 Elasti ...

  6. Elasticsearch 关键字:索引,类型,字段,索引状态,mapping,文档

    1. 索引(_index)索引:说的就是数据库的名字.我这个说法是对应到咱经常使用的数据库. 结合es的插件 head 来看. 可以看到,我这个地方,就有这么几个索引,索引就是数据库,后面是这个数据库 ...

  7. 详解在Word文档中常见的各种公式编辑问题

    正常情况下,我们在安装完成MathType之后会直接加载在Word文档中,Word文档中的MathType比较复杂,新手操作遇到麻烦也是常有的事,今天就来给大家详解下Word文档中常见的MathTyp ...

  8. Elastic Stack 笔记(四)Elasticsearch5&period;6 索引及文档管理

    博客地址:http://www.moonxy.com 一.前言 在 Elasticsearch 中,对文档进行索引等操作时,既可以通过 RESTful 接口进行操作,也可以通过 Java 也可以通过 ...

  9. SharePoint 2010 文档管理系列之准备篇

    前言:很早自己就想写一个系列的文章,但是不知道写什么,最近在QQ群里,好多人说在做文档管理,其实文档管理也是SharePoint的一个很不错的功能点,自己想了想,也想多学习点东西,所以写这个主题吧,今 ...

随机推荐

  1. -bash&colon; &sol;bin&sol;rm&colon; Argument list too long的解决办法

    当目录下文件太多时,用rm删除文件会报错: -bash: /bin/rm: Argument list too long 提示文件数目太多. 解决的办法是使用如下命令: rm -fr ls 输出所有的 ...

  2. access手工注入

    1.判断数据类型 and exists (select * from msysobjects) >0 and exists (select * from sysobjects) >0 一般 ...

  3. 从官方下载 Bootstrap 版本 并写 第一个页面

    从官方下载  Bootstrap 版本 页面内容参考自 http://www.cnblogs.com/sanjuantianshu/p/3935120.html bootstrap-3.2.0.zip ...

  4. 【阿里云产品公测】在Laravel4框架中使用阿里云OCS缓存

    作者:阿里云用户 supechina Laravel 是我最近用得非常多而且越用就越喜欢的一款PHP框架,由于没有向下兼容的历史包袱,完全面向对象的风格,借助 Facades 优雅的 IoC Cont ...

  5. 兼容PHP和Java的des加密解密代码分享

    这篇文章主要介绍了兼容PHP和Java的des加密解密代码分享,适合如服务器是JAVA语言编写,客户端是PHP编写,并需要des加密解密的情况,需要的朋友可以参考下 作为一个iOS工程师来解决安卓的问 ...

  6. Eclipse debug经常使用基本技巧

    1.F5单步调试,步入,进入函数体内部 2.F6单步调试.步过.不进入函数体 3.F7返回 4.F8运行到最后 5.退出时.右键点击右上角Debug选择退出就可以 $(function () { $( ...

  7. SQL Cast()函数

    sql cast()函数 2010-09-17 13:30:26| 分类: Sql | 标签:sql case() 函数 |字号大中小 订阅 (1).CAST()函数的参数是一个表达式,它包括用AS关 ...

  8. &lbrack;C&plus;&plus;&rsqb;const修饰符

    Date: 2014-1-1 Summary: const 修饰符笔记 Contents: 1.const 修饰符 声明一个常量数据类型 , 在编译时就确定数据类型 2.const 与 指针 一般情况 ...

  9. HashTable

    算法打基础——HashTable 这一节主要讲很多方面非常重要的hash table等问题. 由于平时很少用到这些,基本都忘了... 怎样快速的在内存中插入.删除.和搜索呢? 这就需要哈希表了 这一节 ...

  10. CentOS 7 rpm安装jdk

    RPM 安装jdk1.8.0_111 ,查询系统自带的jdk rpm -qa | grep java 查询结果如下: [root@bogon ~]# rpm -qa | grep java javap ...