将巨大的xml数据写入文件的最佳方法？

I'm currently exporting a database table with huge data (100000+ records) into an xml file using XmlTextWriter class and I'm writing directly to a file on the physical drive.

我目前正在使用XmlTextWriter类将包含大量数据(100000条记录)的数据库表导出到xml文件中,并且我直接写入物理驱动器上的文件。

_XmlTextWriterObject = new XmlTextWriter(_xmlFilePath, null);

While my code runs ok, my question is that is it the best approach? Should I instead write the whole xml in memory stream first and then write the xml document in physical file from memory stream? And what are the effects on memory/ performance in both cases?

虽然我的代码运行正常但我的问题是它是最好的方法吗?我应该先将整个xml写入内存流中,然后将xml文档从内存流中写入物理文件中吗?在这两种情况下,对内存/性能的影响是什么?

EDIT

Sorry that I could not actually convey what I meant to say.Thanks Ash for pointing out. I will indeed be using XmlTextWriter but I meant to say whether to pass a physical file path string to the XmlTextWriter constructor (or, as John suggested, to the XmlTextWriter.Create() method) or use stream based api. My current code looks like the following:

对不起,我实际上无法表达我的意思。谢谢Ash指出。我确实会使用XmlTextWriter,但我想说是否将物理文件路径字符串传递给XmlTextWriter构造函数(或者,如John建议的那样,传递给XmlTextWriter.Create()方法)或使用基于流的api。我当前的代码如下所示:

XmlWriter objXmlWriter = XmlTextWriter.Create(new BufferedStream(new FileStream(@"C:\test.xml", FileMode.Create, System.Security.AccessControl.FileSystemRights.Write, FileShare.None, 1024, FileOptions.SequentialScan)), new XmlWriterSettings { Encoding = Encoding.Unicode, Indent = true, CloseOutput = true });
using (objXmlWriter)
{
   //writing xml contents here
}

3 个解决方案

#1

While my code runs ok, my question is that is it the best approach?

As mentioned and your update, XmlWriter.Create is fine.

如上所述和您的更新,XmlWriter.Create很好。

Or should I write the whole xml in memory stream first and then write the xml document in physical file from memory stream?

Do you have the memory to write the entire file in-memory? If you do then that approach will be faster, otherwise stream it using a FileStream which will take care of it for you.

你有内存将整个文件写入内存吗?如果您这样做,那么该方法将更快,否则使用FileStream流式传输它将为您处理它。

And what are the effects on memory/ performance in both cases?

Reading the entire XML file in will use more memory, and spike the processor to start with. Streaming to disk will use more processor. But you'll need to be using a huge file for this to be noticeable given even desktop hardware now. If you're worried about the size increasing even more in the future, stick to the FileStream technique to future proof it.

读取整个XML文件将使用更多内存,并启动处理器。流式传输到磁盘将使用更多的处理器。但是,即使是桌面硬件,你也需要使用一个巨大的文件才能引起注意。如果您担心未来的大小会增加更多,请坚持使用FileStream技术进行未来验证。

#2

The rule of thumb is to use XmlWriter when the document need only be written and not worked with in memory, and to use XmlDocument (or the DOM) where you do need to work with it in memory.

经验法则是当文档只需要编写而不能在内存中使用时使用XmlWriter,并使用XmlDocument(或DOM),你需要在内存中使用它。

Remember though, XmlWriter implements IDisposable, so do the following:

但请记住,XmlWriter实现了IDisposable,因此请执行以下操作:

using (XmlWriter _XmlTextWriterObject = XmlWriter.Create(_xmlFilePath))
{
   // Code to do the write here
}

#3

As John Saunders mentioned it is better to use XmlWriter.Create(). That is the recommendation from the MSDN. The XmlWriter.Create() method can also take an XmlWriterSettings object. There you can customize your behavior quite a bit. If you don't need validation and character checking then you can turn it off and get a bit more speed. For example

正如John Saunders所说,最好使用XmlWriter.Create()。这是MSDN的建议。 XmlWriter.Create()方法也可以使用XmlWriterSettings对象。在那里你可以自定义你的行为。如果您不需要验证和字符检查,那么您可以将其关闭并获得更快的速度。例如

XmlWriterSettings settings = new XmlWriterSettings();
settings.CheckCharacters = false;
using (XmlWriter writer = XmlWriter.Create("path", settings))
{
    //writing code
    writer.Flush();
}

Otherwise I think everything is okay.

否则我觉得一切都还好。