· Monit 简介
Monit是一个轻量级(500KB)跨平台的用来监控Unix/linux系统的开源工具。部署简单,并且不依赖任何第三方程序、插件或者库。
Monit可以监控服务器进程、文件、文件系统、网络状态(HTTP/SMTP等协议)、远程主机、服务器资源变化等等。 并且可以设定资源变化后需要做的动作,比如服务失败后自动重启,邮件告警等等。
Monit内置了WEB UI,可以一目了然地了解监控项的情况。Monit是监控本机服务的工具,M/Monit是其配套产品用以对Monit统一管理,但是M/Monit并不是开源的,需要购买。
本文我们介绍在CentOS7.3上部署Monit用来监控进程状态,并实现服务失败后自动重启以及邮件告警通知的功能。监控其他资源的方法在默认配置文件中都有举例这里不一一介绍。
Monit官网:https://mmonit.com/monit
Monit手册:https://mmonit.com/monit/documentation/monit.html
· Monit 安装
# rpm -ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# yum install -y monit
·Monit 常用命令
monit -t # 配置文件检测
monit # 启动monit daemon
monit -c /var/monit/monitrc # 启动monit daemon时指定配置文件
monit reload # 重新加载配置文件
monit status # 查看所有监控项务状态
monit status nginx # 查看nginx服务状态
monit stop all # 停止所有服务,这里需要注意的是,如果开启了自动重启功能,停止某个被监控的服务必须用monit stop xxx,若用系统命令停止服务,Monit会自动再把服务起来。
monit stop nginx # 停止nginx服务
monit start all # 启动所有服务
monit start nginx # 启动nginx服务
monit -V # 查看版本
·Monit 配置文件
/etc/monit.conf 主配置文件
/etc/monit.d/ 各项服务单独配置文件路径,在主配置文件中将其include进来。
/etc/monit.conf 举例说明:
配置文件关键字:'if', 'and', 'with(in)', 'has', 'us(ing|e)', 'on(ly)', 'then', 'for', 'of'
###############################################################################
## Monit control file
###############################################################################
##
## Comments begin with a '#' and extend through the end of the line. Keywords
## are case insensitive. All path's MUST BE FULLY QUALIFIED, starting with '/'.
##
## Below you will find examples of some frequently used statements. For
## information about the control file and a complete list of statements and
## options, please have a look in the Monit manual.
##
##
###############################################################################
## Global section
###############################################################################
##
## Start Monit in the background (run as a daemon):
#
# 设置检测周期30sset daemon 30 # check services at 30 seconds intervals
# with start delay 240 # optional: delay the first check by 4-minutes (by
# # default Monit check immediately after Monit start)
#
#
## Set syslog logging. If you want to log to a standalone log file instead,
## specify the full path to the log file
#
#设置log路径,这里默认记录到syslogset logfile syslog
#
#
## Set the location of the Monit lock file which stores the process id of the
## running Monit instance. By default this file is stored in $HOME/.monit.pid
#
# set pidfile /var/run/monit.pid
#
## Set the location of the Monit id file which stores the unique id for the
## Monit instance. The id is generated and stored on first Monit start. By
## default the file is placed in $HOME/.monit.id.
#
# set idfile /var/.monit.id
#
## Set the location of the Monit state file which saves monitoring states
## on each cycle. By default the file is placed in $HOME/.monit.state. If
## the state file is stored on a persistent filesystem, Monit will recover
## the monitoring state across reboots. If it is on temporary filesystem, the
## state will be lost on reboot which may be convenient in some situations.
#
# set statefile /var/.monit.state
#
## Set the list of mail servers for alert delivery. Multiple servers may be
## specified using a comma separator. If the first mail server fails, Monit
# will use the second mail server in the list and so on. By default Monit uses
# port 25 - it is possible to override this with the PORT option.
#
# set mailserver mail.bar.baz, # primary mailserver
# backup.bar.baz port 10025, # backup mailserver on port 10025
# localhost # fallback relay
#
# 设置邮件服务器用来发送邮件告警通知 set mailserver mail.abcd.so
#
## By default Monit will drop alert events if no mail servers are available.
## If you want to keep the alerts for later delivery retry, you can use the
## EVENTQUEUE statement. The base directory where undelivered alerts will be
## stored is specified by the BASEDIR option. You can limit the queue size
## by using the SLOTS option (if omitted, the queue is limited by space
## available in the back end filesystem).
#
# set eventqueue
# basedir /var/monit # set the base directory where events will be stored
# slots 100 # optionally limit the queue size
#
#
## Send status and events to M/Monit (for more informations about M/Monit
## see http://mmonit.com/). By default Monit registers credentials with
## M/Monit so M/Monit can smoothly communicate back to Monit and you don't
## have to register Monit credentials manually in M/Monit. It is possible to
## disable credential registration using the commented out option below.
## Though, if safety is a concern we recommend instead using https when
## communicating with M/Monit and send credentials encrypted.
#
# set mmonit http://monit:monit@192.168.1.10:8080/collector
# # and register without credentials # Don't register credentials
#
#
## Monit by default uses the following format for alerts if the the mail-format
## statement is missing::
## --8<--
# 设置邮件告警通知格式
set mail-format {
from: monit@$HOST
subject: monit alert -- $EVENT $SERVICE
message: $EVENT Service $SERVICE
Date: $DATE
Action: $ACTION
Host: $HOST
Description: $DESCRIPTION
Your faithful employee,
Monit
}
## --8<--
##
## You can override this message format or parts of it, such as subject
## or sender using the MAIL-FORMAT statement. Macros such as $DATE, etc.
## are expanded at runtime. For example, to override the sender, use:
#
# set mail-format { from: monit@foo.bar }
#
#
## You can set alert recipients whom will receive alerts if/when a
## service defined in this file has errors. Alerts may be restricted on
## events by using a filter as in the second example below.
#
# 设置邮件告警通知人,Monit默认会通知monit进程本身的变化情况,如果不想收到monit进程自身的通知,加上but not on {instance}配置 set alert weian@abcd.so but not on { instance } # receive all alerts
#
## Do not alert when Monit starts, stops or performs a user initiated action.
## This filter is recommended to avoid getting alerts for trivial cases.
#
# set alert weian@abcd.so
#
#
## Monit has an embedded HTTP interface which can be used to view status of
## services monitored and manage services from a web interface. The HTTP
## interface is also required if you want to issue Monit commands from the
## command line, such as 'monit status' or 'monit restart service' The reason
## for this is that the Monit client uses the HTTP interface to send these
## commands to a running Monit daemon. See the Monit Wiki if you want to
## enable SSL for the HTTP interface.
#
# 设置UI界面访问信息set httpd port 2812 and
use address 10.2.2.28 # only accept connection from localhost
# allow localhost # allow localhost to connect to the server and
allow admin:monit # require user 'admin' with password 'monit'
###############################################################################
## Services
###############################################################################
##
## Check general system resources such as load average, cpu and memory
## usage. Each test specifies a resource, conditions and the action to be
## performed should a test fail.
#
# check system $HOST
# if loadavg (1min) > 4 then alert
# if loadavg (5min) > 2 then alert
# if cpu usage > 95% for 10 cycles then alert
# if memory usage > 75% then alert
# if swap usage > 25% then alert
#
#
## Check if a file exists, checksum, permissions, uid and gid. In addition
## to alert recipients in the global section, customized alert can be sent to
## additional recipients by specifying a local alert handler. The service may
## be grouped using the GROUP option. More than one group can be specified by
## repeating the 'group name' statement.
#
# check file apache_bin with path /usr/local/apache/bin/httpd
# if failed checksum and
# expect the sum 8f7f419955cefa0b33a2ba316cba3659 then unmonitor
# if failed permission 755 then unmonitor
# if failed uid root then unmonitor
# if failed gid root then unmonitor
# alert security@foo.bar on {
# checksum, permission, uid, gid, unmonitor
# } with the mail-format { subject: Alarm! }
# group server
#
#
## Check that a process is running, in this case Apache, and that it respond
## to HTTP and HTTPS requests. Check its resource usage such as cpu and memory,
## and number of children. If the process is not running, Monit will restart
## it by default. In case the service is restarted very often and the
## problem remains, it is possible to disable monitoring using the TIMEOUT
## statement. This service depends on another service (apache_bin) which
## is defined above.
#
# check process apache with pidfile /usr/local/apache/logs/httpd.pid
# start program = "/etc/init.d/httpd start" with timeout 60 seconds
# stop program = "/etc/init.d/httpd stop"
# if cpu > 60% for 2 cycles then alert
# if cpu > 80% for 5 cycles then restart
# if totalmem > 200.0 MB for 5 cycles then restart
# if children > 250 then restart
# if loadavg(5min) greater than 10 for 8 cycles then stop
# if failed host www.tildeslash.com port 80 protocol http
# and request "/somefile.html"
# then restart
# if failed port 443 type tcpssl protocol http
# with timeout 15 seconds
# then restart
# if 3 restarts within 5 cycles then unmonitor
# depends on apache_bin
# group server
#
# 监控进程可以通过上面监控pid文件的方式,当没有pid文件时,可以通过MATCHING正则表达式来匹配进程。
# 测试一个进程是否匹配来自命令行使用的模式monit procmatch "regex-pattern",这将列出匹配或不匹配的所有进程,regex模式。
# 我们这里监控了包含shop-pad-server字段的进程,并指明了启动以及停止的命令,这样在进程因故断掉后,Monit会自动重启进程。
# 同时若进程ID变动,会发送邮件通知到之前指定的收件人。
check process shop-pad-server with MATCHING shop-pad-server
start program = "/usr/bin/nohup /home/azureuser/pad-server/run.sh > /home/azureuser/pad-server/nohup.out 2>&1 &"
stop program = "/usr/bin/ps -ef | /usr/bin/grep shop-pad| /usr/bin/grep -v grep | /usr/bin/awk '{print $2}' | xargs kill"
if changed pid then alert
#
## Check filesystem permissions, uid, gid, space and inode usage. Other services,
## such as databases, may depend on this resource and an automatically graceful
## stop may be cascaded to them before the filesystem will become full and data
## lost.
#
# check filesystem datafs with path /dev/sdb1
# start program = "/bin/mount /data"
# stop program = "/bin/umount /data"
# if failed permission 660 then unmonitor
# if failed uid root then unmonitor
# if failed gid disk then unmonitor
# if space usage > 80% for 5 times within 15 cycles then alert
# if space usage > 99% then stop
# if inode usage > 30000 then alert
# if inode usage > 99% then stop
# group server
#
#
## Check a file's timestamp. In this example, we test if a file is older
## than 15 minutes and assume something is wrong if its not updated. Also,
## if the file size exceed a given limit, execute a script
#
# check file database with path /data/mydatabase.db
# if failed permission 700 then alert
# if failed uid data then alert
# if failed gid data then alert
# if timestamp > 15 minutes then alert
# if size > 100 MB then exec "/my/cleanup/script" as uid dba and gid dba
#
#
## Check directory permission, uid and gid. An event is triggered if the
## directory does not belong to the user with uid 0 and gid 0. In addition,
## the permissions have to match the octal description of 755 (see chmod(1)).
#
# check directory bin with path /bin
# if failed permission 755 then unmonitor
# if failed uid 0 then unmonitor
# if failed gid 0 then unmonitor
#
#
## Check a remote host availability by issuing a ping test and check the
## content of a response from a web server. Up to three pings are sent and
## connection to a port and an application level network check is performed.
#
# check host myserver with address 192.168.1.1
# if failed ping then alert
# if failed port 3306 protocol mysql with timeout 15 seconds then alert
# if failed port 80 protocol http
# and request /some/path with content = "a string"
# then alert
#
#
## Check a network link status (up/down), link capacity changes, saturation
## and bandwidth usage.
#
# check network public with interface eth0
# if failed link then alert
# if changed link then alert
# if saturation > 90% then alert
# if download > 10 MB/s then alert
# if total upload > 1 GB in last hour then alert
#
#
## Check custom program status output.
#
# check program myscript with path /usr/local/bin/myscript.sh
# if status != 0 then alert
#
#
###############################################################################
## Includes
###############################################################################
##
## It is possible to include additional configuration parts from other files or
## directories.
#
# include /etc/monit.d/*
#
# 导入其他单项服务的监控配置
# Include all files from /etc/monit.d/
include /etc/monit.d/*
· 通知周期设置
Monit默认情况下如果一个服务失败只发送一个通知: alert foo@bar
如果您希望在服务保持处于失败状态时每十个周期通知一次,您可以使用: alert foo@bar with reminder on 10 cycles
同样,如果您想在每个失败的周期获得通知,您可以使用: alert foo@bar with reminder on 1 cycle
要禁止某些用户和服务的警报,可以在服务检查的局部配置里添加语句: noalert mail-address
· 服务检测周期设置
可以使用every语句修改服务检查计划。
1.轮询周期倍数
EVERY [number] CYCLES
2.Cron-style
EVERY [cron]
# [cron]
# * * * * *
# 分 时 日 月 周
3.与Cron-style相反(do-not-check)
NOT EVERY [cron]
示例:
示例1:每两个周期检查一次
check process nginx with pidfile /var/run/nginx.pid
every 2 cycles
示例2:在上午8点到下午7点之间检查每个工作日
check program checkOracleDatabase
with path /var/monit/programs/checkoracle.pl
every "* 8-19 * * 1-5"
示例3:在星期日0AM到3AM之间不要在备份窗口中运行检查,否则运行具有常规轮询周期频率的检查。
check process mysqld with pidfile /var/run/mysqld.pid
not every "* 0-3 * * 0"
注意不要使用特定的分钟,因为Monit可能不会在那分钟运行。
· WEB UI界面
参考:
http://www.cnblogs.com/52fhy/p/6412547.html
http://blog.csdn.net/senlin1202/article/details/54291176
https://mmonit.com/monit/documentation/monit.html