I have a python script which is querying a MySQL server on a shared linux host. For some reason, queries to MySQL often return a "server has gone away" error:
我有一个python脚本,它在共享的linux主机上查询MySQL服务器。出于某种原因,对MySQL的查询经常返回“服务器已经消失”错误:
_mysql_exceptions.OperationalError: (2006, 'MySQL server has gone away')
If you try the query again immediately afterwards, it usually succeeds. So, I'd like to know if there's a sensible way in python to try to execute a query, and if it fails, to try again, up to a fixed number of tries. Probably I'd want it to try 5 times before giving up altogether.
如果您之后立即再次尝试查询,它通常会成功。所以,我想知道在python中是否有一种合理的方法来尝试执行查询,如果失败,再次尝试,最多可以尝试一定数量的查询。可能我希望它在放弃之前尝试5次。
Here's the kind of code I have:
这是我的代码类型:
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
Clearly I could do it by having another attempt in the except clause, but that's incredibly ugly, and I have a feeling there must be a decent way to achieve this.
显然,我可以通过在except子句中再次尝试来做到这一点,但这非常难看,而且我觉得必须有一个体面的方法来实现这一点。
10 个解决方案
#1
72
How about:
怎么样:
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
attempts = 0
while attempts < 3:
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
break
except MySQLdb.Error, e:
attempts += 1
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
#2
73
Building on Dana's answer, you might want to do this as a decorator:
在Dana的回答基础上,您可能希望将其作为装饰者:
def retry(howmany):
def tryIt(func):
def f():
attempts = 0
while attempts < howmany:
try:
return func()
except:
attempts += 1
return f
return tryIt
Then...
然后...
@retry(5)
def the_db_func():
# [...]
Enhanced version that uses the decorator
module
import decorator, time
def retry(howmany, *exception_types, **kwargs):
timeout = kwargs.get('timeout', 0.0) # seconds
@decorator.decorator
def tryIt(func, *fargs, **fkwargs):
for _ in xrange(howmany):
try: return func(*fargs, **fkwargs)
except exception_types or Exception:
if timeout is not None: time.sleep(timeout)
return tryIt
Then...
然后...
@retry(5, MySQLdb.Error, timeout=0.5)
def the_db_func():
# [...]
To install the decorator
module:
要安装装饰器模块:
$ easy_install decorator
#3
9
UPDATE: there is a better maintained fork of the retrying library called tenacity, which supports more features and is in general more flexible.
更新:有一个名为tenacity的重试库的更好维护分支,它支持更多功能,并且通常更灵活。
Yes, there is the retrying library, which has a decorator that implements several kinds of retrying logic that you can combine:
是的,有重试库,它有一个装饰器,可以实现几种可以组合的重试逻辑:
Some examples:
一些例子:
@retry(stop_max_attempt_number=7)
def stop_after_7_attempts():
print "Stopping after 7 attempts"
@retry(wait_fixed=2000)
def wait_2_s():
print "Wait 2 second between retries"
@retry(wait_exponential_multiplier=1000, wait_exponential_max=10000)
def wait_exponential_1000():
print "Wait 2^x * 1000 milliseconds between each retry,"
print "up to 10 seconds, then 10 seconds afterwards"
#4
7
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
for i in range(3):
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
break
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
#5
6
Like S.Lott, I like a flag to check if we're done:
像S.Lott一样,我喜欢用旗子来检查我们是否完成了:
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
success = False
attempts = 0
while attempts < 3 and not success:
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
success = True
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
attempts += 1
#6
5
I'd refactor it like so:
我会像这样重构它:
def callee(cursor):
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
def caller(attempt_count=3, wait_interval=20):
""":param wait_interval: In seconds."""
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
for attempt_number in range(attempt_count):
try:
callee(cursor)
except MySQLdb.Error, e:
logging.warn("MySQL Error %d: %s", e.args[0], e.args[1])
time.sleep(wait_interval)
else:
break
Factoring out the callee
function seems to break up the functionality so that it's easy to see the business logic without getting bogged down in the retry code.
考虑到被调用者函数似乎打破了功能,因此很容易看到业务逻辑而不会陷入重试代码中。
#7
1
1.Definition:
1.定义:
def try_three_times(express):
att = 0
while att < 3:
try: return express()
except: att += 1
else: return u"FAILED"
2.Usage:
2.词汇使用:
try_three_times(lambda: do_some_function_or_express())
I use it for parse html context.
我用它来解析html上下文。
#8
0
This is my generic solution:
这是我的通用解决方案:
class TryTimes(object):
''' A context-managed coroutine that returns True until a number of tries have been reached. '''
def __init__(self, times):
''' times: Number of retries before failing. '''
self.times = times
self.count = 0
def __next__(self):
''' A generator expression that counts up to times. '''
while self.count < self.times:
self.count += 1
yield False
def __call__(self, *args, **kwargs):
''' This allows "o() calls for "o = TryTimes(3)". '''
return self.__next__().next()
def __enter__(self):
''' Context manager entry, bound to t in "with TryTimes(3) as t" '''
return self
def __exit__(self, exc_type, exc_val, exc_tb):
''' Context manager exit. '''
return False # don't suppress exception
This allows code like the following:
这允许代码如下:
with TryTimes(3) as t:
while t():
print "Your code to try several times"
Also possible:
也可能:
t = TryTimes(3)
while t():
print "Your code to try several times"
This can be improved by handling exceptions in a more intuitive way, I hope. Open to suggestions.
我希望通过以更直观的方式处理异常来改善这一点。接受建议。
#9
0
def successful_transaction(transaction):
try:
transaction()
return True
except SQL...:
return False
succeeded = any(successful_transaction(transaction)
for transaction in repeat(transaction, 3))
#10
-1
how about this,
这个怎么样,
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
attempt = 5 #set it to desired number of attempts
while attempt:
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
attempt = False #if `try` statement success, end loop
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
attempt -= 1 #every `except`, decrease the number of attempt by 1
this way you can save a memory of a variable. and don't have to worry about ruining whole exception handling logic. (e.g. skipping finally
statement)
这样你就可以保存变量的内存。并且不必担心破坏整个异常处理逻辑。 (例如,跳过最后声明)
Python handles 0
as False
. and rest of int as True
.
You can also set attempt
to a negative number and work it as an endless loop.
Python将0处理为False。和其余的int为True。您还可以将尝试设置为负数,并将其作为无限循环进行操作。
#1
72
How about:
怎么样:
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
attempts = 0
while attempts < 3:
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
break
except MySQLdb.Error, e:
attempts += 1
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
#2
73
Building on Dana's answer, you might want to do this as a decorator:
在Dana的回答基础上,您可能希望将其作为装饰者:
def retry(howmany):
def tryIt(func):
def f():
attempts = 0
while attempts < howmany:
try:
return func()
except:
attempts += 1
return f
return tryIt
Then...
然后...
@retry(5)
def the_db_func():
# [...]
Enhanced version that uses the decorator
module
import decorator, time
def retry(howmany, *exception_types, **kwargs):
timeout = kwargs.get('timeout', 0.0) # seconds
@decorator.decorator
def tryIt(func, *fargs, **fkwargs):
for _ in xrange(howmany):
try: return func(*fargs, **fkwargs)
except exception_types or Exception:
if timeout is not None: time.sleep(timeout)
return tryIt
Then...
然后...
@retry(5, MySQLdb.Error, timeout=0.5)
def the_db_func():
# [...]
To install the decorator
module:
要安装装饰器模块:
$ easy_install decorator
#3
9
UPDATE: there is a better maintained fork of the retrying library called tenacity, which supports more features and is in general more flexible.
更新:有一个名为tenacity的重试库的更好维护分支,它支持更多功能,并且通常更灵活。
Yes, there is the retrying library, which has a decorator that implements several kinds of retrying logic that you can combine:
是的,有重试库,它有一个装饰器,可以实现几种可以组合的重试逻辑:
Some examples:
一些例子:
@retry(stop_max_attempt_number=7)
def stop_after_7_attempts():
print "Stopping after 7 attempts"
@retry(wait_fixed=2000)
def wait_2_s():
print "Wait 2 second between retries"
@retry(wait_exponential_multiplier=1000, wait_exponential_max=10000)
def wait_exponential_1000():
print "Wait 2^x * 1000 milliseconds between each retry,"
print "up to 10 seconds, then 10 seconds afterwards"
#4
7
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
for i in range(3):
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
break
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
#5
6
Like S.Lott, I like a flag to check if we're done:
像S.Lott一样,我喜欢用旗子来检查我们是否完成了:
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
success = False
attempts = 0
while attempts < 3 and not success:
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
success = True
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
attempts += 1
#6
5
I'd refactor it like so:
我会像这样重构它:
def callee(cursor):
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
def caller(attempt_count=3, wait_interval=20):
""":param wait_interval: In seconds."""
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
for attempt_number in range(attempt_count):
try:
callee(cursor)
except MySQLdb.Error, e:
logging.warn("MySQL Error %d: %s", e.args[0], e.args[1])
time.sleep(wait_interval)
else:
break
Factoring out the callee
function seems to break up the functionality so that it's easy to see the business logic without getting bogged down in the retry code.
考虑到被调用者函数似乎打破了功能,因此很容易看到业务逻辑而不会陷入重试代码中。
#7
1
1.Definition:
1.定义:
def try_three_times(express):
att = 0
while att < 3:
try: return express()
except: att += 1
else: return u"FAILED"
2.Usage:
2.词汇使用:
try_three_times(lambda: do_some_function_or_express())
I use it for parse html context.
我用它来解析html上下文。
#8
0
This is my generic solution:
这是我的通用解决方案:
class TryTimes(object):
''' A context-managed coroutine that returns True until a number of tries have been reached. '''
def __init__(self, times):
''' times: Number of retries before failing. '''
self.times = times
self.count = 0
def __next__(self):
''' A generator expression that counts up to times. '''
while self.count < self.times:
self.count += 1
yield False
def __call__(self, *args, **kwargs):
''' This allows "o() calls for "o = TryTimes(3)". '''
return self.__next__().next()
def __enter__(self):
''' Context manager entry, bound to t in "with TryTimes(3) as t" '''
return self
def __exit__(self, exc_type, exc_val, exc_tb):
''' Context manager exit. '''
return False # don't suppress exception
This allows code like the following:
这允许代码如下:
with TryTimes(3) as t:
while t():
print "Your code to try several times"
Also possible:
也可能:
t = TryTimes(3)
while t():
print "Your code to try several times"
This can be improved by handling exceptions in a more intuitive way, I hope. Open to suggestions.
我希望通过以更直观的方式处理异常来改善这一点。接受建议。
#9
0
def successful_transaction(transaction):
try:
transaction()
return True
except SQL...:
return False
succeeded = any(successful_transaction(transaction)
for transaction in repeat(transaction, 3))
#10
-1
how about this,
这个怎么样,
conn = MySQLdb.connect(host, user, password, database)
cursor = conn.cursor()
attempt = 5 #set it to desired number of attempts
while attempt:
try:
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
# do something with the data
attempt = False #if `try` statement success, end loop
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
attempt -= 1 #every `except`, decrease the number of attempt by 1
this way you can save a memory of a variable. and don't have to worry about ruining whole exception handling logic. (e.g. skipping finally
statement)
这样你就可以保存变量的内存。并且不必担心破坏整个异常处理逻辑。 (例如,跳过最后声明)
Python handles 0
as False
. and rest of int as True
.
You can also set attempt
to a negative number and work it as an endless loop.
Python将0处理为False。和其余的int为True。您还可以将尝试设置为负数,并将其作为无限循环进行操作。