I have this task
我有这个任务
def some_task(self, param):
do_some_stuff_that_might_break_with_models()
normally I could do this:
通常我可以这样做：
def some_task(self, param):
try:
with transaction.atomic():
do_some_stuff_that_might_break_with_models()
except:
self.retry(....)
But by doing so, it seems like if I have a couple workers performing this task it'll lock the db because of the atomic block. But if I get rid of it my try except will get angry. How could I handle this? I've tried preprocessing my input with unique hashes and then dispatching each chunk to its own worker and getting rid of the atomic and try blocks but by doing so I lost the ability to retry/track what tasks failed. I can still see what failed from the admin panel through djcelery, but not within the task itself.
但通过这样做，似乎我有几个工作人员执行此任务，它将因为原子块而锁定数据库。但如果我摆脱它，我的尝试除了会生气。我怎么能处理这个？我尝试用独特的哈希预处理我的输入，然后将每个块分派给它自己的工作者并摆脱原子和try块但是这样做我失去了重试/跟踪哪些任务失败的能力。我仍然可以看到从管理面板到djcelery失败的内容，但不在任务本身内。
1 个解决方案
#1
0
Have you verified that you can lock the DB like this? I'd expect the context manager to exit on any exception, rolling back the transaction: https://docs.python.org/3/reference/datamodel.html#object.exit.
您是否已验证可以像这样锁定数据库？我希望上下文管理器退出任何异常，回滚事务：https：//docs.python.org/3/reference/datamodel.html#object.exit。
If you're using Redis and want to avoid concurrent access entirely, I've had success with python-redis-lock as a distributed lock. You can use it in a blocking mode or not.
如果您正在使用Redis并希望完全避免并发访问，那么我已经成功将python-redis-lock作为分布式锁。您可以在阻止模式下使用它。
#1
0
Have you verified that you can lock the DB like this? I'd expect the context manager to exit on any exception, rolling back the transaction: https://docs.python.org/3/reference/datamodel.html#object.exit.
您是否已验证可以像这样锁定数据库？我希望上下文管理器退出任何异常，回滚事务：https：//docs.python.org/3/reference/datamodel.html#object.exit。
If you're using Redis and want to avoid concurrent access entirely, I've had success with python-redis-lock as a distributed lock. You can use it in a blocking mode or not.
如果您正在使用Redis并希望完全避免并发访问，那么我已经成功将python-redis-lock作为分布式锁。您可以在阻止模式下使用它。