内存限制和非常大的XML文件

I need to parse a very big XML file, with a filesize of 750Mo !

我需要解析一个非常大的XML文件,文件大小为750Mo!

I have meomy limit at 512M

我有512M的meomy限制

ini_set('memory_limit', '512M');

I have no problem to open file under 30Mo, but with 750Mo, I obtain a fatal error

我没有问题在30Mo下打开文件,但是750Mo,我得到一个致命的错误

Fatal error: Allowed memory size of 1677721600 bytes exhausted (tried to allocate 2988843769 bytes)

I do that to open files :

我这样做是为了打开文件:

$fichier = file_get_contents($inputfileName);
$xmlInput = simplexml_load_string(utf8_encode($fichier));

Have you an idea to open this file ?

你有想法打开这个文件?

3 个解决方案

#1

Using the DOM based extensions will take up significantly more memory as the raw XML is because the XML will be parsed completely into a tree structure of nodes. Have a look at XMLReader instead

使用基于DOM的扩展将占用更多的内存,因为原始XML是因为XML将被完全解析为节点的树结构。请看一下XMLReader

The XMLReader extension is an XML Pull parser. The reader acts as a cursor going forward on the document stream and stopping at each node on the way.

XMLReader扩展是一个XML Pull解析器。读取器在文档流上作为光标前进,并在途中停在每个节点上。

and make sure you parse with LIBXML_PARSEHUGE

并确保使用LIBXML_PARSEHUGE进行解析

An alternative would the event-based XMLParser

另一种选择是基于事件的XMLParser

#2

You want a SAX or other event-based xml parser. Google 'php sax parser'.

您需要SAX或其他基于事件的xml解析器。谷歌'php sax解析器'。

#3

For big file, perfect use XMLReader class. But if liked simplexml:

对于大文件,完美使用XMLReader类。但如果喜欢simplexml:

Code: https://github.com/dkrnl/SimpleXMLReader/blob/master/library/SimpleXMLReader.php

Usage example: http://github.com/dkrnl/SimpleXMLReader/blob/master/examples/example1.php

用法示例:http://github.com/dkrnl/SimpleXMLReader/blob/master/examples/example1.php

#1