[operator]ELK6的安装

时间：2020-12-06 09:12:52

找了很久才找到一个博客写得比较全面的，FrankDeng

系统环境：CentOS7

相关软件：node-v10.9.0.tar.gz、kibana-6.4.0-linux-x86_64.tar.gz、logstash-6.4.0.zip、elasticsearch-6.4.0.tar.gz，下载地址

先准备三个节点，我这里在每个系统里的/etc/hosts里都添加下了以下内容，在后面的elasticsearch（es）集群的配置里会用到

192.168.133.177　　CentOS7-ELk-node21

192.168.133.178　　CentOS7-ELk-node22

192.168.133.179　　CentOS7-ELk-node23

这里在node21里进行集群的配置

[root@CentOS7-ELk-node21 ~]# grep -v "^#" /opt/module/elk/config/elasticsearch.yml

cluster.name: mycluster

node.name: CentOS7-ELk-node21

bootstrap.memory_lock: false

network.host: 192.168.133.177

discovery.zen.ping.unicast.hosts: ["192.168.133.177", "192.168.133.178", "192.168.133.179"]

discovery.zen.minimum_master_nodes:

在node22里进行的配置

[root@CentOS7-ELK-node22 ~]# !grep

grep -v "^#" /opt/module/elk/config/elasticsearch.yml

cluster.name: mycluster

node.name: CentOS7-ELk-node22

network.host: 192.168.133.178

discovery.zen.ping.unicast.hosts: ["192.168.133.177", "192.168.133.178", "192.168.133.179"]

discovery.zen.minimum_master_nodes:

在node23里进行的配置

[root@CentOS7-ELK-node23 ~]# !grep

grep -v "^#" /opt/module/elk/config/elasticsearch.yml

cluster.name: mycluster

node.name: CentOS7-ELk-node23

network.host: 192.168.133.179

discovery.zen.ping.unicast.hosts: ["192.168.133.177", "192.168.133.178", "192.168.133.179"]

discovery.zen.minimum_master_nodes:

在node22和node23的节点的elasticsearch.yml的文件里将node.name的值改成自动对应的内容然后直接启动es

[root@CentOS7-ELk-node21 ~]# /opt/module/elk/bin/elasticsearch
[root@CentOS7-ELk-node22 ~]# /opt/module/elk/bin/elasticsearch
[root@CentOS7-ELk-node23 ~]# /opt/module/elk/bin/elasticsearch

使用curl命令进行访问看到如下内容说明启动成功

[root@CentOS7-ELk-node21 ~]# curl http://CentOS7-ELk-node21:9200

{

  "name" : "CentOS7-ELk-node21",

  "cluster_name" : "mycluster",

  "cluster_uuid" : "_na_",

  "version" : {

    "number" : "6.4.0",

    "build_flavor" : "default",

    "build_type" : "tar",

    "build_hash" : "595516e",

    "build_date" : "2018-08-17T23:18:47.308994Z",

    "build_snapshot" : false,

    "lucene_version" : "7.4.0",

    "minimum_wire_compatibility_version" : "5.6.0",

    "minimum_index_compatibility_version" : "5.0.0"

  },

  "tagline" : "You Know, for Search"

}

安装个最常用的head插件，下载地址，因为它实际上是一个node.js工作，所以这先来安装个node.js

[root@CentOS7-ELK-node21 ~]# cd /opt/module/node-v10.9.0/
[root@CentOS7-ELK-node21 node-v10.9.0]# ./configure --prefix=/opt/module/node
[root@CentOS7-ELK-node21 node-v10.9.0]# make && make install

经过一次刺激战场经典局鸡屁股的时间。。。

装完成了之后将如下配置添加到/etc/profie里

export NODE_HOME=/opt/module/node

export PATH=$NODE_HOME/bin:$PATH

生效

[root@CentOS7-ELk-node21 node]# source /etc/profile

到elasticsearch-head压缩包所有的目录，解压它后需要安装grunt，先来设置个国内源后再执行

[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm config set registry https://registry.npm.taobao.org
[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm install -g grunt
/opt/module/node/bin/grunt -> /opt/module/node/lib/node_modules/grunt/bin/grunt
+ grunt@1.0.3
added 96 packages from 60 contributors in 5.424s
[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm install
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)

> phantomjs-prebuilt@2.1.16 install /usr/local/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js

PhantomJS not found on PATH
Download already available at /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
skipped....
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 66 packages from 69 contributors in 8.049s

版本确认一下

[root@CentOS7-ELk-node21 elasticsearch-head-master]# node -v

v10.9.0

[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm -v

6.2.

[root@CentOS7-ELk-node21 elasticsearch-head-master]# grunt -version

grunt-cli v1.2.0

grunt v1.0.1

修改head的源码文件

[root@CentOS7-ELk-node21 elasticsearch-head-master]# vim Gruntfile.js
connect: {

                        server: {

                                options: {

                                        hostname: '0.0.0.0',

                                        port: ,

                                        base: '.',

                                        keepalive: true

                                }

                        }

                }

修改es的配置文件

[root@CentOS7-ELk-node21 elk]# tail - config/elasticsearch.yml

http.cors.enabled: true

http.cors.allow-origin: "*"

重启es,启动head

[root@CentOS7-ELk-node21 elasticsearch-head-master]# grunt server &

[]

[root@CentOS7-ELk-node21 elasticsearch-head-master]# (node:) ExperimentalWarning: The http2 module is an experimental API.

Running "connect:server" (connect) task

Waiting forever...

Started connect web server on http://localhost:9100

在启动中间会遇到类bootstrap checkd failed的报错，在这个错误下面会有错误的详细信息，常见的有两种

ERROR: [] bootstrap checks failed

[]: max file descriptors [] for elasticsearch process is too low, increase to at least []

[]: max virtual memory areas vm.max_map_count [] is too low, increase to at least []

解决方案：

编辑 /etc/security/limits.conf，追加以下内容；

* soft nofile

* hard nofile

此文件修改后需要重新登录用户，才会生效

另外一种：

[]: max virtual memory areas vm.max_map_count [] is too low, increase to at least []

解决方案：

编辑 /etc/sysctl.conf，追加以下内容：

vm.max_map_count=

保存后，执行：

sysctl -p

重新启动

启动好之后访问node21的9100端口

[operator]ELK6的安装

再来一个bigdesk插件

[root@CentOS7-ELk-node21 local]# wget https://github.com/hlstudio/bigdesk/archive/master.zip

[root@CentOS7-ELk-node21 local]# yum install httpd -y

[root@CentOS7-ELk-node21 local]# systemctl start httpd

[root@CentOS7-ELk-node21 local]# systemctl enable httpd

bigdesk是一个master.zip的包，解压后将它移动到/var/www/html下面即可

[root@CentOS7-ELk-node21 local]# mv bigdesk-master/ /var/www/html/

使用浏览器访问：http://192.168.133.177/bigdesk-master/_site/#nodes

[operator]ELK6的安装

可以说相当好看了，再来一发IK的分词

[root@CentOS7-ELk-node21 local]# mv bigdesk-master/ /var/www/html/

[root@CentOS7-ELk-node21 local]# cd /opt/module/elk/bin/

[root@CentOS7-ELk-node21 bin]# su es

[es@CentOS7-ELk-node21 bin]$ ./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.0/elasticsearch-analysis-ik-6.4.0.zip

本人在安装的时候第一次出现Exception in thread "main" java.net.ConnectException: Connection refused (Connection refused),可怜我的上进心，再来一次

-> Downloading https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.0/elasticsearch-analysis-ik-6.4.0.zip

[=================================================] %  

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@     WARNING: plugin requires additional permissions     @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

* java.net.SocketPermission * connect,resolve

See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html

for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y

-> Installed analysis-ik

重启es后,测试IK，

[es@CentOS7-ELk-node21 bin]$ curl -XPUT http://192.168.133.177:9200/index

[operator]ELK6的安装

logstash的安装

[root@CentOS7-ELk-node21 local]# unzip filebeat-6.4.-linux-x86_64.tar.gz

[root@CentOS7-ELk-node21 local]# cd logstash-6.4.

修改logstach的配置文件

[root@CentOS7-ELk-node21 logstash-6.4.]# vim logstash-simple.conf

input { stdin { } }

output {

  elasticsearch { hosts => ["192.168.133.177:9200"] }

}

一般在企业中，filebeat做为input，然后output到es中，这里得先构建离线插件

[root@CentOS7-ELk-node21 bin]# ./logstash-plugin prepare-offline-pack logstash-input-beats

[root@CentOS7-ELk-node21 bin]# ./logstash-plugin install file:///usr/local/logstash-6.4.0/logstash-offline-plugins-6.4.0.zip

修改配置文件

[root@CentOS7-ELk-node21 config]# vim logstash-sample.conf

input {

  beats {

    port =>

  }

}

output {

  elasticsearch {

    hosts => ["http://192.168.133.177:9200"]

    #stdout { codec => rubydebug }

    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

    manage_template => false

    document_type => "%{[@metadata][type]}"

    #user => "elastic"

    #password => "changeme"

  }

}

启动logstash

[root@CentOS7-ELk-node21 logstash-6.4.]# bin/logstash -f config/logstash-sample.conf &

logstash只是一个日志引擎，日志其实都是通过filebeat上报的，我这里在node22上安装filebeat，解压后进行filebeat目录后，修改配置文件

[root@CentOS7-ELK-node22 filebeat-6.4.-linux-x86_64]# vim filebeat.yml

- type: log

  enabled: true

  paths:

    - /var/log/nginx/*.log    //如果没有安装nignx的可以这写这一段，这里有设置了上报日志的参数，我的目的只要有日志生成就可以了

- type: log

  # Change to true to enable this input configuration.

  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.

  paths:

    - /var/log/*.log

output.logstash:

  # The Logstash hosts

  hosts: ["192.168.133.177:5044"]    //注意这里

启动filebeat

[root@CentOS7-ELK-node22 filebeat-6.4.-linux-x86_64]# ./filebeat -e -c filebeat.yml

好的，再安装个kibana，这就全部结束了

[root@CentOS7-ELk-node21 kibana-6.4.-linux-x86_64]# cd

/usr/local/kibana-6.4.-linux-x86_64

[root@CentOS7-ELk-node21 kibana-6.4.-linux-x86_64]# grep -v "^#\|^$" config/kibana.yml

server.host: "192.168.133.177"

elasticsearch.url: "http://192.168.133.177:9200"

启动kibana

[root@CentOS7-ELk-node21 kibana-6.4.-linux-x86_64]# bin/kibana &

看到端口占用5601即可使用浏览器浏览

[operator]ELK6的安装

在第一次浏览的时候会要我们设置索引的关键字，在设置栏的下面也会有相应的提示

[operator]ELK6的安装

设置好之后再回到Discover的菜单就可以看到你的日志了

[operator]ELK6的安装

相关文章