Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

时间:2023-12-04 19:54:38

前提:

Elasticsearch-2.4.3的下载(图文详解)

Elasticsearch-2.4.3的单节点安装(多种方式图文详解)

  我这里,以192.168.80.10(HadoopMaster)、192.168.80.11(HadoopSlave1)、192.168.80.12(HadoopSlave2)三台机器,来安装ES的3节点集群部署。

  具体的,3集群搭建安装基础,请移步,先看

hadoop-2.6.0.tar.gz的集群搭建(3节点)

hadoop-2.6.0-cdh5.4.5.tar.gz(CDH)的3节点集群搭建

  更多,请移步,扩展看

http://www.cnblogs.com/zlslch/category/913076.html

ElasticSearch 规划-集群规划

      Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 ElasticSearch 规划-集群规划

        Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

ElasticSearch 规划-用户规划

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

ElasticSearch 规划-目录规划

   软件目录:/home/hadoop/app/

数据目录:/home/hadoop/data/elasticsearch-2.4.3/data

日志目录:/home/hadoop/data/elasticsearch-2.4.3/datalog

   进程目录:/home/hadoop/data/elasticsearch-2.4.3/pid

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

    三台均要去做。



 

ElasticSearch 集群环境检查-时钟同步

1.设置本地时间

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

2.集群时间日期同步NTP

yum install ntp

ntpdate pool.ntp.org

ElasticSearch 集群环境检查-hosts文件检查

配置ElasticSearch集群ip与hostname之间的对应关系

vi /etc/hosts

192.168.80.10   HadoopMaster

192.168.80.11   HadoopSlave1

192.168.80.12   HadoopSlave2

ElasticSearch 集群环境检查-禁用防火墙

查看防火墙状态

service iptables status

关闭防火墙

chkconfig iptables off //永久关闭防火墙

service iptables stop //临时关闭防火墙

ElasticSearch jdk安装

下载解压

配置环境变量

这个很简单,不多赘述

Elasticsearch-2.4.3的3节点安装的前提步骤

1、HadoopMaster

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:EE:5E:5B
inet addr:192.168.80.10 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feee:5e5b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:258 errors:0 dropped:0 overruns:0 frame:0
TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:24027 (23.4 KiB) TX bytes:7192 (7.0 KiB) lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b) [hadoop@HadoopMaster ~]$ ssh HadoopSlave1
Last login: Wed Feb 22 00:34:08 2017 from 192.168.80.1
[hadoop@HadoopSlave1 ~]$ ssh HadoopSlave2
Last login: Wed Feb 22 00:34:09 2017 from 192.168.80.1
[hadoop@HadoopSlave2 ~]$   证明,ssh免密码登录是可以的。

2、HadoopSlave1

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:D8:AA:6A
inet addr:192.168.80.11 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fed8:aa6a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:249 errors:0 dropped:0 overruns:0 frame:0
TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:23321 (22.7 KiB) TX bytes:5876 (5.7 KiB) lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b) [hadoop@HadoopSlave1 ~]$ ssh HadoopMaster
Last login: Wed Feb 22 00:34:06 2017 from 192.168.80.1
[hadoop@HadoopMaster ~]$ ssh HadoopSlave2
Last login: Wed Feb 22 00:34:45 2017 from hadoopslave1
[hadoop@HadoopSlave2 ~]$

3、HadoopSlave2

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:FA:65:4A
inet addr:192.168.80.12 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fefa:654a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:199 errors:0 dropped:0 overruns:0 frame:0
TX packets:119 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:23401 (22.8 KiB) TX bytes:17008 (16.6 KiB) lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b) [hadoop@HadoopSlave2 ~]$ ssh HadoopMaster
Last login: Wed Feb 22 00:38:54 2017 from hadoopslave1
[hadoop@HadoopMaster ~]$ ssh HadoopSlave1
Last login: Wed Feb 22 00:34:33 2017 from hadoopmaster
[hadoop@HadoopSlave1 ~]$

Elasticsearch-2.4.3的3节点安装的过程步骤

  1、上传elasticsearch-2.4.3.tar.gz压缩包

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster app]$ pwd
/home/hadoop/app
[hadoop@HadoopMaster app]$ ll
total 20
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ rz [hadoop@HadoopMaster app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 app]$ pwd
/home/hadoop/app
[hadoop@HadoopSlave1 app]$ ll
total 20
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$ rz [hadoop@HadoopSlave1 app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 app]$ pwd
/home/hadoop/app
[hadoop@HadoopSlave2 app]$ ll
total 16
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$ rz

[hadoop@HadoopSlave2 app]$ ll
total 26720
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$

  2、解压缩elasticsearch-2.4.3.tar.gz压缩包

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ tar -zxvf elasticsearch-2.4.3.tar.gz

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$ tar -zxvf elasticsearch-2.4.3.tar.gz

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 app]$ ll
total 26720
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$ tar -zxvf elasticsearch-2.4.3.tar.gz

  3、删除elasticsearch-2.4.3.tar.gz压缩包

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster app]$ ll
total 26728
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ rm elasticsearch-2.4.3.tar.gz
[hadoop@HadoopMaster app]$ ll
total 24
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 app]$ ll
total 26728
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$ rm elasticsearch-2.4.3.tar.gz
[hadoop@HadoopSlave1 app]$ ll
total 24
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 app]$ ll
total 26724
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:38 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$ rm elasticsearch-2.4.3.tar.gz
[hadoop@HadoopSlave2 app]$ ll
total 20
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:38 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$

  4、解读认识ES的目录结构

  这里,以HadoopMaster为例

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ ll
total 44
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 bin
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 config
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 lib
-rw-rw-r--. 1 hadoop hadoop 11358 Aug 24 00:46 LICENSE.txt
drwxrwxr-x. 5 hadoop hadoop 4096 Dec 8 00:41 modules
-rw-rw-r--. 1 hadoop hadoop 150 Aug 24 00:46 NOTICE.txt
-rw-rw-r--. 1 hadoop hadoop 8700 Aug 24 00:46 README.textile
[hadoop@HadoopMaster elasticsearch-2.4.3]$

  5、修改配置文件

  进入$ES_HOME/config

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ cd config/
[hadoop@HadoopMaster config]$ pwd
/home/hadoop/app/elasticsearch-2.4.3/config
[hadoop@HadoopMaster config]$ ll
total 8
-rw-rw-r--. 1 hadoop hadoop 3192 Aug 24 00:46 elasticsearch.yml
-rw-rw-r--. 1 hadoop hadoop 2571 Aug 24 00:46 logging.yml
[hadoop@HadoopMaster config]$ vim elasticsearch.yml

  当然,这个配置文件,直接可以下载到本地,用Notepad++等软件来打开。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

# network.host: 192.168.0.1

# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# cluster.name: my-application
# node.name: node-1
# path.data: /path/to/data
# path.logs: /path/to/logs 改为 network.host: 0.0.0.0 (当然这里,可以写为192.168.80.10)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 120s
client.transport.ping_timeout: 60s
discovery.zen.ping.unicast.hosts: ["192.168.80.10","192.168.80.11", "192.168.80.12"]
cluster.name: escluster
node.name: node-0
path.data: /home/hadoop/data/elasticsearch-2.4.3/data (当然若为了更专业,比如实现负载均衡,挂载不同盘上,则可以用多个目录,用逗号隔开句好,这里为了学习只写一个)
path.logs: /home/hadoop/data/elasticsearch-2.4.3/datalog
其实,这里自行去设置。这里牵扯到单播和多播

  注意,这些一定要顶格。具体,更多详细,见

elasticsearch-.yml(中文配置详解)

  对其他两台也要进行同样的配置文件修改工作。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

# network.host: 192.168.0.1

# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# cluster.name: my-application
# node.name: node-1
# path.data: /path/to/data
# path.logs: /path/to/logs 改为 network.host: 0.0.0.0 (当然,这里可以写为192.168.80.11)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 120s
client.transport.ping_timeout: 60s
discovery.zen.ping.unicast.hosts: ["192.168.80.10","192.168.80.11", "192.168.80.12"]
cluster.name: escluster
node.name: node-1
path.data: /home/hadoop/data/elasticsearch-2.4.3/data (当然若为了更专业,比如实现负载均衡,挂载不同盘上,则可以用多个目录,用逗号隔开句好,这里为了学习只写一个)
path.logs: /home/hadoop/data/elasticsearch-2.4.3/
datalog 其实,这里自行去设置。这里牵扯到单播和多播

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

# network.host: 192.168.0.1

# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# cluster.name: my-application
# node.name: node-1
# path.data: /path/to/data
# path.logs: /path/to/logs 改为

network.host:
0.0.0.0 (当然这里,可以写为192.168.80.12)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 120s
client.transport.ping_timeout: 60s
discovery.zen.ping.unicast.hosts: ["192.168.80.10","192.168.80.11", "192.168.80.12"]
cluster.name: escluster
node.name: node-2
path.data: /home/hadoop/data/elasticsearch-2.4.3/data (当然若为了更专业,比如实现负载均衡,挂载不同盘上,则可以用多个目录,用逗号隔开句好,这里为了学习只写一个)
path.logs: /home/hadoop/data/elasticsearch-2.4.3/
datalog 其实,这里自行去设置。这里牵扯到单播和多播

  或者用scp命令,进行分发,这个很简单!

ElasticSearch 插件安装(含 head、kopf、marvel、shield和watcher插件安装)

  安装marvel插件(所有节点都需要安装)(在线安装)

    bin/plugin install license

    bin/plugin install marvel-agent

  安装head插件(选择一个节点安装即可)(在线安装)(其实安装一个节点就可以看到整个集群的了)(我这里就3个节点都安装了)

    bin/plugin install mobz/elasticsearch-head

  6下载并安装插件(含 head、kopf、marvel、shield和watcher插件安装)

Elasticsearch之插件介绍及安装

Elasticsearch之head插件安装之后的浏览详解

Elasticsearch之kopf插件安装之后的浏览详解

  我这里,以离线方式来安装es插件之Head Plugin Kopf Plugin。

      以在线方式来安装es插件之和marvel插件。

安装 head和kopf插件:

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster app]$ pwd
/home/hadoop/app
[hadoop@HadoopMaster app]$ ll
total 24
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ rz [hadoop@HadoopMaster app]$ rz [hadoop@HadoopMaster app]$ ll
total 3092
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 908862 Jan 10 11:38 elasticsearch-head-master.zip
-rw-r--r--. 1 hadoop hadoop 2228252 Jan 10 11:38 elasticsearch-kopf-master.zip
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$

  

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ ll
total 44
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 bin
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 02:08 config
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 lib
-rw-rw-r--. 1 hadoop hadoop 11358 Aug 24 00:46 LICENSE.txt
drwxrwxr-x. 5 hadoop hadoop 4096 Dec 8 00:41 modules
-rw-rw-r--. 1 hadoop hadoop 150 Aug 24 00:46 NOTICE.txt
-rw-rw-r--. 1 hadoop hadoop 8700 Aug 24 00:46 README.textile
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/elasticsearch-head-master.zip
-> Installing from file:/home/hadoop/app/elasticsearch-head-master.zip...
Plugins directory [/home/hadoop/app/elasticsearch-2.4.3/plugins] does not exist. Creating...
Trying file:/home/hadoop/app/elasticsearch-head-master.zip ...
Downloading .........DONE
Verifying file:/home/hadoop/app/elasticsearch-head-master.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
Installed head into /home/hadoop/app/elasticsearch-2.4.3/plugins/head
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/elasticsearch-kopf-master.zip
-> Installing from file:/home/hadoop/app/elasticsearch-kopf-master.zip...
Trying file:/home/hadoop/app/elasticsearch-kopf-master.zip ...
Downloading ......................DONE
Verifying file:/home/hadoop/app/elasticsearch-kopf-master.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
Installed kopf into /home/hadoop/app/elasticsearch-2.4.3/plugins/kopf
[hadoop@HadoopMaster elasticsearch-2.4.3]$

  同理,HadoopSlave1和HadoopSlave2也如上述步骤。这里,不多赘述。

  安装marvel插件:

  (所有节点都需要安装)(HadoopMaster、HadoopSlave1和Hadoopslave2)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip checksums if available ...
Downloading .DONE
Installed license into /home/hadoop/app/elasticsearch-2.4.3/plugins/license
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install marvel-agent
-> Installing marvel-agent...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.4.3/marvel-agent-2.4.3.zip ...
Downloading ..........DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.4.3/marvel-agent-2.4.3.zip checksums if available ...
Downloading .DONE
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission setFactory
* javax.net.ssl.SSLPermission setHostnameVerifier
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y
Installed marvel-agent into /home/hadoop/app/elasticsearch-2.4.3/plugins/marvel-agent
[hadoop@HadoopMaster elasticsearch-2.4.3]$

同理,HadoopSlave1和HadoopSlave2也如上述步骤。这里,不多赘述。

  安装shield插件:

(所有节点都需要安装)(HadoopMaster、HadoopSlave1和Hadoopslave2)

  默认情况下elasticsearch可以不受限制访问,如果在外网的情况下就不安全了,所以,elastic官方提供了shield插件,可以实现权限控制,但是不好的地方就是这个插件是收费的,可以免费试用一个月。那我就带你来试试吧!

  我告诉你一个小技巧:等到30天之后,卸载 shield,再重新安装就是。该如何卸载,后面我也有写。

  Shield 插件必须安装在集群中的每一个节点,并且每一个节点在安装完成之后必须要重启才能生效。

  我这里,是3节点集群,所以HadoopMaster、HadoopSlave1和HadoopSlave2都安装shield插件。

  注意:你必须保证Shield的版本和你运行的Elasticsearch的版本保持一致。

Java 7 or later
Elasticsearch 2.4.3
Elasticsearch License 2.4.3 plugin

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip checksums if available ...
Downloading .DONE
ERROR: plugin directory /home/hadoop/app/elasticsearch-2.4.3/plugins/license already exists. To update the plugin, uninstall it first using 'remove license' command
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install shield

    当然,若你的网速不好的话,也可以选择离线来安装。即,先下载好。

https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip

https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/shield/2.4.3/shield-2.4.3.zip

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/shield-2.4.3.zip
-> Installing from file:/home/hadoop/app/shield-2.4.3.zip...
Trying file:/home/hadoop/app/shield-2.4.3.zip ...
Downloading .......................DONE
Verifying file:/home/hadoop/app/shield-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y
Installed shield into /home/hadoop/app/elasticsearch-2.4.3/plugins/shield
[hadoop@HadoopMaster elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 同理,HadoopSlave1和HadoopSlave2也如上述步骤。

  然后,默认情况下es并没有禁用自动创建索引库功能。

  假设,若你在以后时,如果你在es中禁用了自动创建索引库的功能。那么你需要在elasticsearch.yml中配置action.auto_create_index 以便允许Shield去创建.security 索引库。action.auto_create_index: .security

  这里,我不赘述。

  然后,启动/重启es

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

安装shield插件之后,如何来使用呢?(HadoopMaster、HadoopSlave1、HadoopSlave2)

  创建用户:

    用户级别分为以下三种: 

      (1)admin:可以对就所有集群或者索引库执行操作

      (2)power_user:可以监控集群并且对所有索引库执行操作

      (3)user:可以对所有索引库执行read操作

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  创建用户命令:【执行之后会提示用户设置密码】

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

bin/shield/esusers useradd es_admin -r admin

解释:
useradd:表示添加用户
es_admin:表示要新建的用户名称
-r:指定权限信息
admin:给新建的用户赋予admin权限

  注意:当提示输入密码的时候,给新用户设置一个新密码,密码必须至少包含6个字符。比如我这里设置为123456

  这也是,为什么后续,编程java api时,需要用到的地方。见如下:

使用java api操作es需要注意的地方(Elasticsearch开启shield插件之后)

  访问es:-u es_admin 表示使用用户es_admin进行访问

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ curl -u es_admin -XGET 'http://192.168.80.10:9200/'
Enter host password for user 'es_admin':
{
"name" : "node-0",
"cluster_name" : "escluster",
"cluster_uuid" : "gBwdRD7iRRyC6TRji7ohFw",
"version" : {
"number" : "2.4.3",
"build_hash" : "d38a34e7b75af4e17ead16f156feffa432b22be3",
"build_timestamp" : "2016-12-07T16:28:56Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
"tagline" : "You Know, for Search"
}
[hadoop@HadoopMaster elasticsearch-2.4.3]$
curl -u es_admin -XGET 'http://localhost:9200/'

或
curl -u es_admin -XGET 'http://192.168.80.10:9200/'

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ bin/shield/esusers useradd es_admin -r admin
Enter new password:
Retype new password:
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ curl -u es_admin -XGET 'http://192.168.80.11:9200/'
Enter host password for user 'es_admin':
{
"name" : "node-1",
"cluster_name" : "escluster",
"cluster_uuid" : "gBwdRD7iRRyC6TRji7ohFw",
"version" : {
"number" : "2.4.3",
"build_hash" : "d38a34e7b75af4e17ead16f156feffa432b22be3",
"build_timestamp" : "2016-12-07T16:28:56Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
"tagline" : "You Know, for Search"
}
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/shield/esusers useradd es_admin -r admin
Enter new password:
Retype new password:
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ curl -u es_admin -XGET 'http://192.168.80.12:9200/'
Enter host password for user 'es_admin':
{
"name" : "node-2",
"cluster_name" : "escluster",
"cluster_uuid" : "gBwdRD7iRRyC6TRji7ohFw",
"version" : {
"number" : "2.4.3",
"build_hash" : "d38a34e7b75af4e17ead16f156feffa432b22be3",
"build_timestamp" : "2016-12-07T16:28:56Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
"tagline" : "You Know, for Search"
}
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  访问的会提示输入刚才设置的密码。(我刚才输入的密码是123456,用户名是es_admin)。

  这样以后再访问es的时候就需要输入用户名和密码了。

kibana中也能使用shield

  具体见,https://www.elastic.co/guide/en/shield/current/kibana.html

  以后写上,

如果您的安全要求更复杂,你可以这样做:

  1、定义和使用细粒度访问控制的自定义角色

  2、使用LDAP或Active Directory集成,或使用证书的认证。

  3、使用IP过滤来自特定IP地址或地址范围的请求

  具体,见https://www.elastic.co/guide/en/shield/current/ip-filtering.html

目前是试用期30天(商业),如何卸载shield?

  如果不想使用shield,需要移除掉。

    第一步:停止es

    第二步:从es中移除shield插件

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/plugin remove shield

    第三步:启动es.

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ jps
2394 Jps
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/elasticsearch -d
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ jps
2428 Jps
2415 Elasticsearch

  这样做的目的是,为了再次重新使用30天,目前,只能这样个傻瓜办法了。

 

 安装watcher插件:

  (所有节点都需要安装)(HadoopMaster、HadoopSlave1和Hadoopslave2)

  watcher插件可以提供elasticsearch集群的监控提醒功能。

   你需要在你的集群中所有节点安装license和watcher插件。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip checksums if available ...
Downloading .DONE
ERROR: plugin directory /home/hadoop/app/elasticsearch-2.4.3/plugins/license already exists. To update the plugin, uninstall it first using 'remove license' command
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install watcher
-> Installing watcher...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/watcher/2.4.3/watcher-2.4.3.zip ...

  可以通过上述网址,去下载到。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  当然,照样也是,可以离线方式来安装的。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/watcher-2.4.3.zip
-> Installing from file:/home/hadoop/app/watcher-2.4.3.zip...
Trying file:/home/hadoop/app/watcher-2.4.3.zip ...
Downloading ................DONE
Verifying file:/home/hadoop/app/watcher-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y
Installed watcher into /home/hadoop/app/elasticsearch-2.4.3/plugins/watcher
[hadoop@HadoopMaster elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/watcher-2.4.3.zip
-> Installing from file:/home/hadoop/app/watcher-2.4.3.zip...
Trying file:/home/hadoop/app/watcher-2.4.3.zip ...
Downloading ................DONE
Verifying file:/home/hadoop/app/watcher-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y
Installed watcher into /home/hadoop/app/elasticsearch-2.4.3/plugins/watcher
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/watcher-2.4.3.zip
-> Installing from file:/home/hadoop/app/watcher-2.4.3.zip...
Trying file:/home/hadoop/app/watcher-2.4.3.zip ...
Downloading ................DONE
Verifying file:/home/hadoop/app/watcher-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y
Installed watcher into /home/hadoop/app/elasticsearch-2.4.3/plugins/watcher
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$

  默认这一步不需要操作,因为默认情况下es并没有禁用自动创建索引库功能。

  如果你在es中禁用了自动创建索引库的功能。那么你需要在elasticsearch.yml中配置action.auto_create_index 以便允许watcher去创建.watches, .triggered_watches, and .watcher-history* 索引库。

  如果你已经安装了shield,你必须允许shield创建.security索引库

  这里,我不赘述。

  然后,启动/重启es

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  验证watcher插件后的效果

curl -XGET 'http://localhost:9200/_watcher/stats?pretty'

或
curl -XGET 'http://192.168.80.10:9200/_watcher/stats?pretty'

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ curl -XGET 'http://192.168.80.10:9200/_watcher/stats?pretty'
{
"error" : {
"root_cause" : [ {
"type" : "security_exception",
"reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
}
} ],
"type" : "security_exception",
"reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
}
},
"status" : 401
}
[hadoop@HadoopMaster elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ curl -XGET 'http://192.168.80.11:9200/_watcher/stats?pretty'
{
"error" : {
"root_cause" : [ {
"type" : "security_exception",
"reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
}
} ],
"type" : "security_exception",
"reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
}
},
"status" : 401
}
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ curl -XGET 'http://192.168.80.12:9200/_watcher/stats?pretty'
{
"error" : {
"root_cause" : [ {
"type" : "security_exception",
"reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
}
} ],
"type" : "security_exception",
"reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
}
},
"status" : 401
}
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

暂时,放这里。

监控功能:

  1:监控错误日志

    具体见,https://www.elastic.co/guide/en/watcher/current/watch-log-data.html

  2:监控集群监控状态

    具体见,https://www.elastic.co/guide/en/watcher/current/watch-cluster-status.html

面以监控集群状态为例演示:

  1:创建一个每10秒运行一次的调度器

curl -XPUT 'http://192.168.80.10:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
}
}'

  2:给这个调度器增加一个input,把集群的健康状态添加到watcher中

curl -XPUT 'http://192.168.80.10:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "192.168.80.100",
"port" : 9200,
"path" : "/_cluster/health"
}
}
}
}'

   3:给这个wacher添加一个条件,判断集群的状态是否为red

curl -XPUT 'http://192.168.80.100:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "192.168.80.100",
"port" : 9200,
"path" : "/_cluster/health"
}
}
},
"condition" : {
"compare" : {
"ctx.payload.status" : { "eq" : "red" }
}
}
}'

  4:给watcher设置触发任务,当满足条件的时候,执行。

  一般都是发送邮件。

  所以需要先配置邮箱信息。修改es的配置文件。

  vi conf/elasticsearch.yml
watcher.actions.email.service.account:
work:
profile: qqmail
smtp:
auth: true
starttls.enable: true
host: smtp.qq.com
port: 587
user: 12345677@qq.com
password: 12345678

  添加触发任务【手动把某一个索引库的分片删除一部分可以模拟分片丢失,集群状态会变为red,此任务就会被触发,发送邮件。】

curl -XPUT 'http://192.168.80.100:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "192.168.80.100",
"port" : 9200,
"path" : "/_cluster/health"
}
}
},
"condition" : {
"compare" : {
"ctx.payload.status" : { "eq" : "yellow" }
}
},
"actions" : {
"send_email" : {
"email" : {
"to" : "87654321@qq.com",
"subject" : "Cluster Status Warning",
"body" : "Cluster status is RED"
}
}
}
}'

  验证集群状态,如果集群状态为red,则刚才配置的watcher任务就会执行,发送邮件提醒。

curl -XGET 'http://192.168.80.10:9200/_cluster/health?pretty'

  5:实验结束,可以移除掉这个watcher

curl -XDELETE 'http://192.168.80.10:9200/_watcher/watch/cluster_health_watch'

  目前是试用期30天(商业),如何卸载watcher?

    第一步:停止Elasticsearch.

    第二步:从Elasticsearch中移除watcher:

bin/plugin remove watcher

    第三步:启动Elasticsearch.

  7、启动,验证

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ jps
2020 Jps
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/elasticsearch -d
[hadoop@HadoopMaster elasticsearch-2.4.3]$ jps
2054 Jps
2041 Elasticsearch
[hadoop@HadoopMaster elasticsearch-2.4.3]$

  其实啊,或者通过es的这种后台来启动也是可以的。

bin/elasticsearch  -d  -p /home/hadoop/data/elasticsearch-2.4.3/pid

  输入,http://192.168.80.10:9200/

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  以上是默认的。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  以上是自己修改了es的配置文件得到的。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 其实啊,或者通过es的这种后台来启动也是可以的。

bin/elasticsearch  -d  -p /home/hadoop/data/elasticsearch-2.4.3/pid

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 以上是默认的。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  以上是自己修改了es的配置文件得到的。

 

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 其实啊,或者通过es的这种后台来启动也是可以的。

bin/elasticsearch  -d  -p /home/hadoop/data/elasticsearch-2.4.3/pid

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  以上是默认的。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  以上是自己修改了es的配置文件得到的。

  三个节点的整个。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 

  启动head插件界面

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  注意:每次启动会随机生成一个节点名称 。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  默认情况下,三台机器,内部默认选举,选出哪台是主节点,但是对于我们而言,也可以自己限制出哪台是主节点。

  在这里,要说明一个问题:

  出现了 no_route_to_host_exception: No route to host ,最后查博客得知,是防火墙没关闭,建议作永久关闭,即 chkconfig iptables off 。

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

 节点和节点之间通信 是走9300端口    
 我们通过http请求连接es是走9200端口 
  插件kopf启动界面
 http://192.168.80.12:9200/_plugin/kopf/#!/cluster
Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  对应,

Elasticsearch-2.4.3的3节点安装(多种方式图文详解)(含 head、kopf、marvel、shield和watcher插件安装和使用)

  这里,不多赘述。

  至此,ES的3台机器组件的集群,成功安装!

扩展学习

Elasticsearch集群如何扩容机器?

实例展示elasticsearch集群生态,分片以及水平扩展.