多线程环境中的文件访问策略（Web App）

I have a file which is an XML representation of some data that is taken from a Web service and cached locally within a Web Application. The idea being is that this data is very static, but just might change. So I have set it up to cache to a file, and stuck a monitor against it to check if it has been deleted. Once deleted, the file will be refreshed from its source and rebuilt.

我有一个文件,它是从Web服务获取并在Web应用程序中本地缓存的某些数据的XML表示形式。我们的想法是,这些数据非常静态,但可能会发生变化。所以我已将其设置为缓存到文件,并将监视器卡在其上以检查它是否已被删除。删除后,文件将从其源代码刷新并重建。

I am now running in to problems though, because obviously in a multi-threaded environment it falls over as it is trying to access the data when it is still reading/writing the file.

我现在正在遇到问题,因为很明显在多线程环境中,当它仍然在读取/写入文件时试图访问数据时它会崩溃。

This is confusing me, because I added a object to lock against, and this is always locked during read/write. It was my understanding that attempted access from other threads would be told to "wait" until the lock was released?

这让我感到困惑,因为我添加了一个要锁定的对象,并且在读/写期间始终会锁定它。据我了解,从其他线程尝试访问将被告知“等待”直到锁被释放?

Just to let you know, I am real new to multi-threaded development, so I am totally willing to accept this is a screw up on my part :)

只是为了让你知道,我是多线程开发的新手,所以我完全愿意接受这是我的一个搞砸:)

Am I missing something?

我错过了什么吗?

What is the best file access strategy in a multi-threaded environment?

多线程环境中最好的文件访问策略是什么?

Edit

Sorry - I should have said this is using ASP.NET 2.0 :)

对不起 - 我应该说这是使用ASP.NET 2.0 :)

5 个解决方案

#1

Here is the code that I use to make sure a file is not locked by another process. It's not 100% foolproof, but it gets the job done most of the time:

这是我用来确保文件没有被另一个进程锁定的代码。这不是100%万无一失,但它大部分时间都可以完成工作:

    /// <summary>
    /// Blocks until the file is not locked any more.
    /// </summary>
    /// <param name="fullPath"></param>
    bool WaitForFile(string fullPath)
    {
        int numTries = 0;
        while (true)
        {
            ++numTries;
            try
            {
                // Attempt to open the file exclusively.
                using (FileStream fs = new FileStream(fullPath,
                    FileMode.Open, FileAccess.ReadWrite, 
                    FileShare.None, 100))
                {
                    fs.ReadByte();

                    // If we got this far the file is ready
                    break;
                }
            }
            catch (Exception ex)
            {
                Log.LogWarning(
                   "WaitForFile {0} failed to get an exclusive lock: {1}", 
                    fullPath, ex.ToString());

                if (numTries > 10)
                {
                    Log.LogWarning(
                        "WaitForFile {0} giving up after 10 tries", 
                        fullPath);
                    return false;
                }

                // Wait for the lock to be released
                System.Threading.Thread.Sleep(500);
            }
        }

        Log.LogTrace("WaitForFile {0} returning true after {1} tries",
            fullPath, numTries);
        return true;
    }

Obviously you can tweak the timeouts and retries to suit your application. I use this to process huge FTP files that take a while to be written.

显然,您可以调整超时和重试以适合您的应用程序。我使用它来处理需要一段时间才能编写的大型FTP文件。

#2

If you're locking on a object stored as a static then the lock should work for all threads in the same Application Domain, but perhaps you need to upload a code sample so we can have a look at the offending lines.

如果您锁定存储为静态的对象,那么锁应该适用于同一应用程序域中的所有线程,但是您可能需要上传代码示例,以便我们可以查看有问题的行。

That said, one thought would be to check if IIS is configured to run in Web Garden mode (i.e. more than 1 process executing your application) which would break your locking logic. While you could fix such a situation with a mutex it'd be easier to reconfigure your application to execute in a single process, although you'd be wise to check the performance before and after messing with the web garden settings as it can potentially affect performance.

也就是说,一个想法是检查IIS是否配置为在Web Garden模式下运行(即执行应用程序的进程超过1个),这会破坏您的锁定逻辑。虽然您可以使用互斥锁来修复这种情况,但是在单个进程中重新配置应用程序更容易,尽管在弄乱Web园设置之前和之后检查性能是明智的,因为它可能会影响性能。

#3

You could maybe create the file with a temporary name ("data.xml_TMP"), and when it's ready change the name to what it is supposed to be. That way, no other process will be accessing it before it is ready.

您可以使用临时名称(“data.xml_TMP”)创建文件,并在准备好时将名称更改为应该是的名称。这样,在准备好之前,没有其他进程可以访问它。

#4

OK, I have been working on this and ended up creating a stress-test module to basically hammer the crap out of my code from several threads (See Related Question).

好的,我一直在研究这个问题并最终创建了一个压力测试模块,基本上可以从几个线程中剔除我的代码(参见相关问题)。

It was much easier from this point on to find holes in my code. It turns out that my code wasn't actually far off, but there was a certain logic path that it could enter in to which basically caused read/write operations to stack up, meaning if they didn't get cleared in time, it would go boom!

从这一点开始,在我的代码中找到漏洞要容易得多。事实证明我的代码实际上并不遥远,但是它可以进入某个逻辑路径,这基本上导致读/写操作堆积起来,这意味着如果它们没有及时清除,它会去热潮!

Once I took that out, ran my stress test again, all worked fine!

一旦我拿出来,再次进行压力测试,一切正常!

So, I didn't really do anything special in my file access code, just ensured I used lock statements where appropriate (i.e. when reading or writing).

所以,我在文件访问代码中没有做任何特别的事情,只是确保我在适当的时候使用了锁定语句(即在读或写时)。

#5

How about using AutoResetEvent to communicate between threads? I created a console app which creates roughly 8 GB file in createfile method and then copy that file in main method

如何使用AutoResetEvent在线程之间进行通信?我创建了一个控制台应用程序,它在createfile方法中创建大约8 GB的文件,然后在main方法中复制该文件

 static AutoResetEvent waitHandle = new AutoResetEvent(false);
    static string filePath=@"C:\Temp\test.txt";
    static string fileCopyPath=@"C:\Temp\test-copy.txt";
    static void Main(string[] args)
    {
        Console.WriteLine("in main method");
        Console.WriteLine();
        Thread thread = new Thread(createFile);
        thread.Start();

        Console.WriteLine("waiting for file to be processed ");
        Console.WriteLine();
        waitHandle.WaitOne();
        Console.WriteLine();

        File.Copy(filePath, fileCopyPath);
        Console.WriteLine("file copied ");

    }


    static void createFile()
    {

        FileStream fs= File.Create(filePath);            
        Console.WriteLine("start processing a file "+DateTime.Now);
        Console.WriteLine();
        using (StreamWriter sw = new StreamWriter(fs))
        {
            for (long i = 0; i < 300000000; i++)
            {
                sw.WriteLine("The value of i is " + i);

            }
        }
        Console.WriteLine("file processed " + DateTime.Now);
        Console.WriteLine();

        waitHandle.Set();
    }

#1