Elasticsearch+Kibana+Logstash安装

时间：2020-12-27 20:30:28

安装环境：

[root@node- src]# cat /etc/redhat-release

CentOS Linux release 7.5. (Core)

安装之前关闭防火墙 firewalld 和 selinux：

[root@node- logs]# systemctl stop firewalld

[root@node- logs]# setenforce

安装流程：

Kibana->Elasticsearch->Logstash

一、安装运行所需的Java环境，Elasticsearch、Logstash依赖于java环境,使用官方的二进制包解压安装，先下载java linux 64位tar.gz包，java 1.8的下载链接：

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

配置JAVA环境：

[root@node- src]# cd /usr/local/src

[root@node- src]# tar xf jdk-8u191-linux-x64.tar.gz

[root@node- src]# mv jdk1..0_191 /usr/local

#用全路径验证java是否安装成功

/usr/local/jdk1..0_191/bin/java -version

#配置java环境变量

vim /etc/profile加入

export JAVA_HOME=/usr/local/jdk1..0_191/

export PATH=$PATH:$JAVA_HOME/bin

export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$CLASSPATH

#环境变量生效

source /etc/profile

#java版本查看

[root@node- ~]# java -version

java version "1.8.0_191"

Java(TM) SE Runtime Environment (build 1.8.0_191-b12)

Java HotSpot(TM) -Bit Server VM (build 25.191-b12, mixed mode)

安装Kibana：

#kibana下载地址(kibana主要用来展现数据，它本身不存储数据)

https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-linux-x86_64.tar.gz

#准备工作，添加elk用户，用elk用户来启动elk

useradd elk

usermod -s /sbin/nologin elk    #不让elk用户来登录系统

#解压安装kibana:

tar -zxf kibana-6.2.-linux-x86_64.tar.gz

mv kibana-6.2.-linux-x86_64 /usr/local/kibana-6.2.

#kibana配置文件

vim /usr/local/kibana-6.2./config/kibana.yml修改：

server.port:

server.host: "0.0.0.0"(监听在所有网卡，有风险)

#elasticsearch.url: "http://localhost:9200" (默认是连接elasticsearch的9200端口)

#elasticsearch.username: "user" (配置连接elasticsearch的用户名和密码)

#elasticsearch.password: "pass"

#把kibana目录改为elk用户

chown -R elk:elk /usr/local/kibana-6.2./

#新增启动脚本vim /usr/local/kibana-6.2./bin/start.sh

nohup /usr/local/kibana-6.2./bin/kibana >>/tmp/kibana.log >>/tmp/kibana.log &

chmod a+x /usr/local/kibana-6.2./bin/start.sh

#用普通用户启动

su -s /bin/bash elk '/usr/local/kibana-6.2.3/bin/start.sh'

访问kibana，如有防火墙需要放开tcp 5601端口

Nginx限制访问kibana：

默认的kibana是没有任何的权限控制，先把kibana改到监听127.0.0.，借助nginx来限制访问

:借助nginx来限制访问，控制源ip的访问

worker_processes  ;

events {

    worker_connections  ;

}

http {

    include       mime.types;

    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                      '$status $body_bytes_sent "$http_referer" '

                      '"$http_user_agent" "$http_x_forwarded_for"';

    sendfile        on;

    keepalive_timeout  ;

    server {

       listen       ;

       access_log  /usr/local/nginx/logs/kibana_access.log main;

       error_log /usr/local/nginx/logs/kibana_error.log error;

       location / {

           allow 127.0.0.1;

           deny all;

           proxy_pass http://127.0.0.1:5601;

       }

    }

}

可以在日志里面找到源ip地址：tail -f /usr/local/nginx/logs/kibana_access.log

: 如果ip经常变化，就会很麻烦。nginx支持简单的用户名密码认证。

location / {

            auth_basic "elk auth";

            auth_basic_user_file /usr/local/nginx/conf/htpasswd;

            proxy_pass http://127.0.0.1:5601;

        }

printf "elk:$(openssl passwd -1 elkpass)\n" >/usr/local/nginx/conf/htpasswd

: nginx源码编译安装脚本

if [ -d "/usr/local/nginx/" ];then

    echo "nginx is install"

    exit

else

    echo "nginx in not install"

fi

for softpack in wget tar gcc gcc-c++ make pcre pcre-devel zlib zlib-devel openssl openssl-devel;do

soft_result=`rpm -qa $softpack`

if [ -z "$soft_result" ];then

echo "${softpack} is not exist,install it"

yum -y install ${softpack}

else

echo "${softpack} is exist"

fi

done

cd /usr/local/src

wget 'http://nginx.org/download/nginx-1.12.2.tar.gz'

tar -zxvf nginx-1.12..tar.gz

cd nginx-1.12.

./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-stream  --with-stream_ssl_module

make

make install

exit 

ln -sf /usr/local/nginx/sbin/nginx /usr/local/bin/

elasticsearch安装配置：

elasticsearch未安装之前，kibana网页上报错，提示找不到elasticsearch。

: elasticsearch的下载地址(elasticsearch主要用来存储数据，供kibana调取并进行展现)

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.tar.gz

解压安装：

cd /usr/local/src/

tar -zxf elasticsearch-6.2..tar.gz

mv elasticsearch-6.2. /usr/local/

: elasticsearch配置

vim /usr/local/elasticsearch-6.2./config/elasticsearch.yml 修改：

path.data: /usr/local/elasticsearch-6.2./data

path.logs: /usr/local/elasticsearch-6.2./logs

network.host: 127.0.0.1

http.port:

bootstrap.memory_lock: false

bootstrap.system_call_filter: false

: 把elasticsearch目录的用户和属主都更新为elk

chown -R elk:elk /usr/local/elasticsearch-6.2./

: 更改jvm的内存限制(看个人配置)

vim /usr/local/elasticsearch-6.2./config/jvm.options

-Xms100M

-Xmx100M

: 编辑elasticsearch启动脚本，使用-d进行后台启动。elasticsearch

vim /usr/local/elasticsearch-6.2./bin/start.sh

/usr/local/elasticsearch-6.2./bin/elasticsearch -d

chmod a+x /usr/local/elasticsearch-6.2./bin/start.sh

: 启动elasticsearch

su -s /bin/bash elk '/usr/local/elasticsearch-6.2.3/bin/start.sh'

观察日志

观察kibana网页，看下还会不会报elasticsearch的错误

: elasticsearch如果监听在非127.0.0.，需要配置内核参数等

network.host: 0.0.0.0

vim /etc/security/limits.conf(处理max file descriptors [] for elasticsearch process is too low, increase to at least [])

* soft nofile

* hard nofile 

vim /etc/security/limits.d/-nproc.conf(处理max number of threads [] for user [elk] is too low, increase to at least [])

*          soft    nproc

*          hard    nproc     

sysctl.conf添加(处理max virtual memory areas vm.max_map_count [] is too low, increase to at least [])

vm.max_map_count =     #需要运行sysctl -p生效

Logstash安装配置：

: logstash的下载地址(用来读取日志，正则分析日志，发送给elasticsearch数据库)

https://artifacts.elastic.co/downloads/logstash/logstash-6.2.3.tar.gz

解压安装：

tar -zxf logstash-6.2..tar.gz

mv logstash-6.2. /usr/local/

ll -h /usr/local/logstash-6.2.

: 更改logstash jvm配置vim /usr/local/logstash-6.2./config/jvm.options

-Xms150M

-Xmx150M

: logstash配置 vim /usr/local/logstash-6.2./config/logstash.conf

input {

  file {

    path => "/usr/local/nginx/logs/kibana_access.log"

  }

}

output {

  elasticsearch {

    hosts => ["http://127.0.0.1:9200"]

  }

}

: logstash的启动脚本：

vim /usr/local/logstash-6.2./bin/start.sh

nohup /usr/local/logstash-6.2./bin/logstash -f /usr/local/logstash-6.2./config/logstash.conf >>/tmp/logstash.log >>/tmp/logstash.log &

chmod a+x  /usr/local/logstash-6.2./bin/start.sh

: 启动logstash

/usr/local/logstash-6.2./bin/start.sh

logstash的启动时间会有点慢，等启动过后查看kibana的界面，会有可以创建索引的地方。

相关文章