如果进程死了，如何编写bash脚本来重启？

I have a python script that'll be checking a queue and performing an action on each item:

我有一个python脚本,它将检查队列并对每个项目执行操作:

# checkqueue.py
while True:
  check_queue()
  do_something()

How do I write a bash script that will check if it's running, and if not, start it. Roughly the following pseudo code (or maybe it should do something like ps | grep?):

如何编写一个bash脚本来检查它是否正在运行,如果没有,则启动它。大致是以下伪代码(或者它应该做类似ps | grep?):

# keepalivescript.sh
if processidfile exists:
  if processid is running:
     exit, all ok

run checkqueue.py
write processid to processidfile

I'll call that from a crontab:

我将从crontab中调用它:

# crontab
*/5 * * * * /path/to/keepalivescript.sh

7 个解决方案

#1

Avoid PID-files, crons, or anything else that tries to evaluate processes that aren't their children.

避免使用PID文件,crons或其他任何试图评估不属于其子级的进程的方法。

There is a very good reason why in UNIX, you can ONLY wait on your children. Any method (ps parsing, pgrep, storing a PID, ...) that tries to work around that is flawed and has gaping holes in it. Just say no.

有一个很好的理由,为什么在UNIX中,你只能等待你的孩子。尝试解决这个问题的任何方法(ps解析,pgrep,存储PID,...)都是有缺陷的,并且在其中有漏洞。拒绝吧。

Instead you need the process that monitors your process to be the process' parent. What does this mean? It means only the process that starts your process can reliably wait for it to end. In bash, this is absolutely trivial.

相反,您需要监控流程的流程是流程的父流程。这是什么意思?这意味着只有启动进程的进程才能可靠地等待它结束。在bash中,这绝对是微不足道的。

until myserver; do
    echo "Server 'myserver' crashed with exit code $?.  Respawning.." >&2
    sleep 1
done

The above piece of bash code runs myserver in an until loop. The first line starts myserver and waits for it to end. When it ends, until checks its exit status. If the exit status is 0, it means it ended gracefully (which means you asked it to shut down somehow, and it did so successfully). In that case we don't want to restart it (we just asked it to shut down!). If the exit status is not 0, until will run the loop body, which emits an error message on STDERR and restarts the loop (back to line 1) after 1 second.

上面的bash代码在until循环中运行myserver。第一行启动myserver并等待它结束。当它结束时,直到检查其退出状态。如果退出状态为0,则表示它正常结束(这意味着您要求它以某种方式关闭,并且它成功完成)。在这种情况下,我们不想重新启动它(我们只是要求它关闭!)。如果退出状态不为0,则直到将运行循环体,该体在STDERR上发出错误消息并在1秒后重新启动循环(返回到第1行)。

Why do we wait a second? Because if something's wrong with the startup sequence of myserver and it crashes immediately, you'll have a very intensive loop of constant restarting and crashing on your hands. The sleep 1 takes away the strain from that.

我们为什么要等一下?因为如果myserver的启动顺序出现问题并且它立即崩溃,那么你将会有一个非常密集的循环,不断重启并崩溃。睡眠1消除了那种压力。

Now all you need to do is start this bash script (asynchronously, probably), and it will monitor myserver and restart it as necessary. If you want to start the monitor on boot (making the server "survive" reboots), you can schedule it in your user's cron(1) with an @reboot rule. Open your cron rules with crontab:

现在您需要做的就是启动此bash脚本(可能是异步),它将监视myserver并根据需要重新启动它。如果要在启动时启动监视器(使服务器“生存”重新启动),可以使用@reboot规则在用户的cron(1)中安排它。使用crontab打开cron规则:

crontab -e

Then add a rule to start your monitor script:

然后添加规则以启动监控脚本:

@reboot /usr/local/bin/myservermonitor

Alternatively; look at inittab(5) and /etc/inittab. You can add a line in there to have myserver start at a certain init level and be respawned automatically.

另外;查看inittab(5)和/ etc / inittab。您可以在其中添加一行以使myserver在某个初始级别启动并自动重新生成。

Edit.

Let me add some information on why not to use PID files. While they are very popular; they are also very flawed and there's no reason why you wouldn't just do it the correct way.

让我添加一些有关不使用PID文件的信息。虽然他们很受欢迎;它们也是非常有缺陷的,没有理由不以正确的方式做到这一点。

Consider this:

PID recycling (killing the wrong process):

PID回收(杀死错误的进程):
- /etc/init.d/foo start: start foo, write foo's PID to /var/run/foo.pid
- A while later: foo dies somehow.
- A while later: any random process that starts (call it bar) takes a random PID, imagine it taking foo's old PID.
- You notice foo's gone: /etc/init.d/foo/restart reads /var/run/foo.pid, checks to see if it's still alive, finds bar, thinks it's foo, kills it, starts a new foo.
PID files go stale. You need over-complicated (or should I say, non-trivial) logic to check whether the PID file is stale, and any such logic is again vulnerable to 1..

PID文件过时了。您需要过度复杂(或者我应该说,非平凡)逻辑来检查PID文件是否过时,并且任何此类逻辑再次容易受到1 ..
What if you don't even have write access or are in a read-only environment?

如果您甚至没有写访问权限或处于只读环境中该怎么办?
It's pointless overcomplication; see how simple my example above is. No need to complicate that, at all.

这是毫无意义的过度复杂化;看看我上面的例子有多简单。根本不需要复杂化。

另请参阅:“正确”执行时,PID文件是否仍有缺陷?

By the way; even worse than PID files is parsing ps! Don't ever do this.

顺便说说;比PID文件更糟糕的是解析ps!不要这样做。

ps is very unportable. While you find it on almost every UNIX system; its arguments vary greatly if you want non-standard output. And standard output is ONLY for human consumption, not for scripted parsing!

ps是非常不可移植的。虽然你几乎在每个UNIX系统上都能找到它;如果你想要非标准输出,它的参数差别很大。标准输出仅供人类使用,而不是脚本解析!

Parsing ps leads to a LOT of false positives. Take the ps aux | grep PID example, and now imagine someone starting a process with a number somewhere as argument that happens to be the same as the PID you stared your daemon with! Imagine two people starting an X session and you grepping for X to kill yours. It's just all kinds of bad.

解析ps会导致很多误报。拿ps aux | grep PID示例,现在想象有人在某个地方启动一个带有数字的进程作为参数,恰好与你盯着你的守护进程的PID相同!想象一下,两个人开始X会话,然后你要用X来杀死你。这只是各种各样的坏事。

If you don't want to manage the process yourself; there are some perfectly good systems out there that will act as monitor for your processes. Look into runit, for example.

如果您不想自己管理流程;有一些非常好的系统可以作为您的流程的监控器。例如,看看runit。

#2

Have a look at monit (http://mmonit.com/monit/). It handles start, stop and restart of your script and can do health checks plus restarts if necessary.

看看monit(http://mmonit.com/monit/)。它处理脚本的启动,停止和重新启动,并且可以执行运行状况检查以及必要时重新启动。

Or do a simple script:

或者做一个简单的脚本:

while true
do
/your/script
sleep 1
done

#3

The easiest way to do it is using flock on file. In Python script you'd do

最简单的方法是在文件中使用flock。在Python脚本中你会这样做

lf = open('/tmp/script.lock','w')
if(fcntl.flock(lf, fcntl.LOCK_EX|fcntl.LOCK_NB) != 0): 
   sys.exit('other instance already running')
lf.write('%d\n'%os.getpid())
lf.flush()

In shell you can actually test if it's running:

在shell中,您可以实际测试它是否正在运行:

if [ `flock -xn /tmp/script.lock -c 'echo 1'` ]; then 
   echo 'it's not running'
   restart.
else
   echo -n 'it's already running with PID '
   cat /tmp/script.lock
fi

But of course you don't have to test, because if it's already running and you restart it, it'll exit with 'other instance already running'

但是当然你不必测试,因为如果它已经运行并且你重新启动它,它将退出'其他实例已经运行'

When process dies, all it's file descriptors are closed and all locks are automatically removed.

当进程死亡时,它的所有文件描述符都将被关闭,并且所有锁都会被自动删除。

#4

You should use monit, a standard unix tool that can monitor different things on the system and react accordingly.

您应该使用monit,这是一个标准的unix工具,可以监视系统上的不同内容并做出相应的反应。

From the docs: http://mmonit.com/monit/documentation/monit.html#pid_testing

来自文档:http://mmonit.com/monit/documentation/monit.html#pid_testing

check process checkqueue.py with pidfile /var/run/checkqueue.pid
       if changed pid then exec "checkqueue_restart.sh"

You can also configure monit to email you when it does do a restart.

您还可以配置monit,以便在重新启动时通过电子邮件发送给您。

#5

if ! test -f $PIDFILE || ! psgrep `cat $PIDFILE`; then
    restart_process
    # Write PIDFILE
    echo $! >$PIDFILE
fi

#6

I've used the following script with great success on numerous servers:

我在许多服务器上使用了以下脚本并取得了巨大成功:

pid=`jps -v | grep $INSTALLATION | awk '{print $1}'`
echo $INSTALLATION found at PID $pid 
while [ -e /proc/$pid ]; do sleep 0.1; done

notes:

It's looking for a java process, so I can use jps, this is much more consistent across distributions than ps

它正在寻找一个java进程,所以我可以使用jps,这在分布上比ps更加一致

$INSTALLATION contains enough of the process path that's it's totally unambiguous

$ INSTALLATION包含足够的进程路径,这是完全明确的

Use sleep while waiting for the process to die, avoid hogging resources :)

在等待进程死亡时使用sleep,避免占用资源:)

This script is actually used to shut down a running instance of tomcat, which I want to shut down (and wait for) at the command line, so launching it as a child process simply isn't an option for me.

这个脚本实际上用于关闭正在运行的tomcat实例,我想在命令行关闭(并等待),所以将它作为子进程启动对我来说根本不是一个选项。

#7

I'm not sure how portable it is across operating systems, but you might check if your system contains the 'run-one' command, i.e. "man run-one". Specifically, this set of commands includes 'run-one-constantly', which seems to be exactly what is needed.

我不确定它在操作系统中的可移植性如何,但您可能会检查您的系统是否包含“run-one”命令,即“man run-one”。具体来说,这组命令包括“run-one-always”,这似乎正是所需要的。

From man page:

从手册页:

run-one-constantly COMMAND [ARGS]

经常运行COMMAND [ARGS]

Note: obviously this could be called from within your script, but also it removes the need for having a script at all.

注意:显然,这可以在您的脚本中调用,但它也不需要拥有脚本。

#1