There are some Java questions similar to this one but none of them had accepted answers that I could find and my situation is a little more specific. I have a program that processes files one at a time as they are fed into a directory. At times, the files may remain locked in use by the program dropping them off for a few seconds after they've are visible to my program. I have no control over that program. In my program I already have error handling to recover if a file is in use when my program tries to read it. It results in the file being skipped entirely. What I want to implement is a retry scheme to see if the file becomes unlocked after a few seconds. In another question here I found a suggestion to use FileUtils.touch()
. This is convenient because I'm already using FileUtils in my project. According to the javadoc, FileUtils.touch()
with throw an IOException
if the file is in use by another process. Sounds great. However, one of the comments under the answer I mentioned warned of a race condition without any explanation. Here is the code I'm considering implementing:
有一些类似于这个问题的Java问题,但没有一个问题接受了我能找到的答案,而且我的情况稍微具体一些。我有一个程序,当它们被送入目录时,一次处理一个文件。有时,程序可能会保持锁定状态,程序在程序可见后将其关闭几秒钟。我无法控制该程序。在我的程序中,我已经有错误处理,以便在我的程序尝试读取文件时使用该文件。它导致文件被完全跳过。我想要实现的是一个重试方案,以查看文件是否在几秒钟后解锁。在另一个问题中,我发现了一个使用FileUtils.touch()的建议。这很方便,因为我已经在我的项目中使用了FileUtils。根据javadoc,如果该文件正由另一个进程使用,则FileUtils.touch()抛出IOException。听起来很棒。但是,我提到的答案中的一条评论警告说没有任何解释的竞争条件。这是我正在考虑实施的代码:
// Implementing Sason's suggestion
int retries = 0;
while (retries < MAX_RETRIES) {
try {
processFile(file);
} catch (IOException ioe) {
// Assumes this is a file in use exception... bad thing?
log.warn("File is in use. Waiting 1 second to retry.");
retries++;
try {
Thread.sleep(1000);
} catch (InterruptedException ie) {
log.warn("Thread interrupted while waiting for file lock to clear.");
break;
}
}
}
I realize that the file could become locked again between the FileUtils.touch() and the processFile()
method call (and I presume that is the race condition warned about in the unaccepted answer I found) but that's fine. The processFile()
method will just handle the lock error like it currently does. To be clear, in this case it's perfectly OK for the file to be open by another process but it cannot be locked.
我意识到文件可能会在FileUtils.touch()和processFile()方法调用之间再次被锁定(我认为这是在我发现的未接受的答案中警告的竞争条件)但是没关系。 processFile()方法将像当前一样处理锁定错误。为了清楚起见,在这种情况下,文件可以由另一个进程打开,但它无法锁定。
Also, while sometimes the file does become unlocked after a second or two, in some instances the program dropping off the files keeps the lock indefinitely. I also need to avoid infinitely trying to reprocess the same file as well as be able to notify the sender when a file could not be processed after the retries have been exhausted.
此外,虽然有时文件确实在一两秒后解锁,但在某些情况下,程序从文件中删除会无限期地保持锁定。我还需要避免无限尝试重新处理同一个文件,以及在重试耗尽后无法处理文件时能够通知发件人。
So am I missing something here? Is there a better/safer way to do this?
我在这里错过了一些东西吗?有没有更好/更安全的方法来做到这一点?
1 个解决方案
#1
3
I would use a SchedueledExecutorService to submit work with some delay that tries to process the file. If the processing fails, post the work again and again until it succeeds.
我会使用SchedueledExecutorService提交一些尝试处理文件的延迟。如果处理失败,请反复发布工作,直到成功为止。
Is FileUtils.touch necessary here? Why not just try to read the file and if it fails retry?
这里需要FileUtils.touch吗?为什么不尝试读取文件,如果重试失败?
The thread.sleep thing is kind of ugly. If processing several files, each one has to use a it's own thread until it is done. A SchedueledExecutorService would be more efficient.
thread.sleep的东西有点难看。如果处理多个文件,每个文件都必须使用它自己的线程,直到完成为止。 SchedueledExecutorService会更有效率。
#1
3
I would use a SchedueledExecutorService to submit work with some delay that tries to process the file. If the processing fails, post the work again and again until it succeeds.
我会使用SchedueledExecutorService提交一些尝试处理文件的延迟。如果处理失败,请反复发布工作,直到成功为止。
Is FileUtils.touch necessary here? Why not just try to read the file and if it fails retry?
这里需要FileUtils.touch吗?为什么不尝试读取文件,如果重试失败?
The thread.sleep thing is kind of ugly. If processing several files, each one has to use a it's own thread until it is done. A SchedueledExecutorService would be more efficient.
thread.sleep的东西有点难看。如果处理多个文件,每个文件都必须使用它自己的线程,直到完成为止。 SchedueledExecutorService会更有效率。