liunx 使用crontab定时任务+shell脚本删除tomcat日志elasticsearch日志索引

时间：2021-02-26 15:55:06

由于过多的日志很占用磁盘空间，今天经理让我写一个脚本删除多余的日志、和es索引

定时删除试根据crontab+shell脚本实现的

crontab配置目录

/var/spool/cron/

该目录下存放的是每个用户（包括root）的crontab任务，文件名以用户名命名（添加命令后会自动生成root）

/etc/cron.d/

这个目录用来存放任何要执行的crontab文件或脚本。

crontab操作步骤

Step-One : 编辑任务脚本【分目录存放】【ex: backup.sh】
Step-Two : 编辑定时文件【命名规则:backup.cron】
Step-Three : crontab命令添加到系统crontab backup.cron
Step-Four : 查看crontab列表 crontab -l

Tomcat日志删除脚本

主要是shell脚本，第一次接触，写不好见谅（说明一下，公司多项目，所以用循环的方式删除）

#!/bin/bash

tomcat1="***-apiservice_beta"

tomcat2="***-gateway_beta"

tomcat3="***-mobile_beta"

tomcat4="***-mobile_preview"

tomcat5="***-portal_beta"

tomcat6="***-portal_preview"

tomcat7="***-portal-remote-freemarker_beta"

tomcat8="***-scheduler_beta"

#设定删除多少天之前的日志

day=

for tomcat  in $tomcat1 $tomcat2 $tomcat3 $tomcat4 $tomcat5 $tomcat6 $tomcat7 $tomcat8

do

    find /opt/tomcat-web-apps/$tomcat/logs -name "host-manager.*" -mtime +$day -exec rm -rf {} \;

    find /opt/tomcat-web-apps/$tomcat/logs -name "localhost.*" -mtime +$day -exec rm -rf {} \;

    find /opt/tomcat-web-apps/$tomcat/logs -name "manager.*" -mtime +$day -exec rm -rf {} \;

    find /opt/tomcat-web-apps/$tomcat/logs -name "catalina.*" -mtime +$day -exec rm -rf {} \;

    find /opt/tomcat-web-apps/$tomcat/logs -name "localhost_access_log*" -mtime + -exec rm -rf {} \;

    find /opt/temp/$tomcat/logs -name "spring*" -mtime +$day -exec rm -rf {} \;

    find /opt/temp/$tomcat/logs/access -name "access*" -mtime +$day -exec rm -rf {} \;

done

elasticsearch日志删除脚本

具体脚本根据实际需求来

#!/bin/bash

elastic_url=127.0.0.1

elastic_port=

day=

user_name=***

user_password=***

search_index1=**-log-gateway-service-main-beta

search_index2=**-log-gateway-service-access-beta

search_index3=**-log-portal-service-main-beta

search_index4=**-log-scheduler-service-main-beta

search_index5=**-log-service-main-beta

#删除早于day天的ES集群的索引

#获取当天时间

date2stamp () {

    date --utc --date "$1" +%s

}

#计算时间差

dateDiff (){

    case $ in

        -s)   sec=;      shift;;

        -m)   sec=;     shift;;

        -h)   sec=;   shift;;

        -d)   sec=;  shift;;

        *)    sec=;;

    esac

    dte1=$(date2stamp $)

    dte2=$(date2stamp $)

    diffSec=$((dte2-dte1))

    if ((diffSec < )); then abs=-; else abs=; fi

    echo $((diffSec/sec*abs))

}

for search_index in $search_index1 $search_index2 $search_index3 $search_index4 $search_index5

do

    for index in $(curl -s "${elastic_url}:${elastic_port}/_cat/indices?v" -u ${user_name}:${user_password} | grep -E "${search_index}-20[0-9][0-9]\-[0-1][0-9]\-[0-3][0-9]" | awk '{ print $3 }';)

    do

      #循环当前索引的日期(这里获取时间的正则根据自己索引的名称来我们索引的格式为：**-log-service-main-beta-2020-03-04) 
　　　 date=$(echo ${index}| grep -Eo "20[0-9][0-9]\-[0-1][0-9]\-[0-3][0-9]")
      #当天日期

      cond=$(date +%Y-%m-%d)

      #时间差

      diff=$(dateDiff -d $date $cond)

      echo -n "${index} (${diff})"

      if [ $diff -gt ${day} ]; then

        echo " yes DELETE"

        # curl -XDELETE "${elastic_url}:${elastic_port}/${index}?pretty"

      else

        echo " no DELETE"

      fi

    done

done

参考地址：

https://*.com/questions/33430055/removing-old-indices-in-elasticsearch#answer-39746705

https://www.jianshu.com/p/838db0269fd0

相关文章