如何逐行读取大型文件

How to read a file line by line in PHP, without completely loading it in memory?

如何在PHP中逐行读取文件，而不将其完全加载到内存中?

My file is too large to open in memory so I always got memory exhaust errors.

我的文件太大，无法在内存中打开，所以我总是会有内存耗尽错误。

The file size is 1 GB.

文件大小为1gb。

13 个解决方案

#1

You can use the fgets() function to read the file line by line:

可以使用fgets()函数逐行读取文件:

$handle = fopen("inputfile.txt", "r");
if ($handle) {
    while (($line = fgets($handle)) !== false) {
        // process the line read.
    }

    fclose($handle);
} else {
    // error opening the file.
} 

#2

if ($file = fopen("file.txt", "r")) {
    while(!feof($file)) {
        $line = fgets($file);
        # do same stuff with the $line
    }
    fclose($file);
}

#3

You can use an object oriented interface class for a file - SplFileObject http://php.net/manual/en/splfileobject.fgets.php (PHP 5 >= 5.1.0)

您可以为文件使用面向对象的接口类—SplFileObject http://php.net/manual/en/splfileobject.fgets.php (PHP 5 >= 5.1.0)

<?php

$file = new SplFileObject("file.txt");

// Loop until we reach the end of the file.
while (!$file->eof()) {
    // Echo one line from the file.
    echo $file->fgets();
}

// Unset the file to call __destruct(), closing the file handle.
$file = null;

#4

Use buffering techniques to read the file.

使用缓冲技术读取文件。

$filename = "test.txt";
$source_file = fopen( $filename, "r" ) or die("Couldn't open $filename");
while (!feof($source_file)) {
    $buffer = fread($source_file, 4096);  // use a buffer of 4KB
    $buffer = str_replace($old,$new,$buffer);
    ///
}

#5

There is a file() function that returns an array of the lines contained in the file.

有一个file()函数，它返回文件中包含的行数组。

foreach(file('myfile.txt') as $line) {
   echo $line. "\n";
}

#6

If you're opening a big file, you probably want to use Generators alongside fgets() to avoid loading the whole file into memory:

如果打开一个大文件，您可能希望在fgets()旁边使用生成器，以避免将整个文件加载到内存中:

/**
 * @return Generator
 */
$fileData = function() {
    $file = fopen(__DIR__ . '/file.txt', 'r');

    if (!$file)
        die('file does not exist or cannot be opened');

    while (($line = fgets($file)) !== false) {
        yield $line;
    }

    fclose($file);
};

Use it like this:

使用它是这样的:

foreach ($fileData() as $line) {
    // $line contains current line
}

This way you can process individual file lines inside the foreach().

这样，您就可以在foreach()中处理单独的文件行。

Note: Generators require >= PHP 5.5

注意:生成器要求>= PHP 5.5

#7

foreach (new SplFileObject(__FILE__) as $line) {
    echo $line;
}

#8

Be careful with the 'while(!feof ... fgets()' stuff, fgets can get an error (returnfing false) and loop forever without reaching the end of file. codaddict was closest to being correct but when your 'while fgets' loop ends, check feof; if not true, then you had an error.

注意“while(!)”feof……fgets()的东西，fgets可以得到一个错误(returnfing false)，并在不到达文件末尾的情况下进行永久循环。瘾君子最接近正确，但当你的“当fgets”循环结束时，检查feof;如果不是，那么你有一个错误。

#9

One of the popular solutions to this question will have issues with the new line character. It can be fixed pretty easy with a simple str_replace.

对这个问题的一种流行的解决方法是使用新的行字符。使用简单的str_replace可以很容易地修复它。

$handle = fopen("some_file.txt", "r");
if ($handle) {
    while (($line = fgets($handle)) !== false) {
        $line = str_replace("\n", "", $line);
    }
    fclose($handle);
}

#10

This how I manage with very big file (tested with up to 100G). And it's faster than fgets()

这就是我如何管理非常大的文件(测试最高可达100G)。比fgets()快

$block =1024*1024;//1MB or counld be any higher than HDD block_size*2
if ($fh = fopen("file.txt", "r")) { 
    $left='';
    while (!feof($fh)) {// read the file
       $temp = fread($fh, $block);  
       $fgetslines = explode("\n",$temp);
       $fgetslines[0]=$left.$fgetslines[0];
       if(!feof($fh) )$left = array_pop($lines);           
       foreach ($fgetslines as $k => $line) {
           //do smth with $line
        }
     }
}
fclose($fh);

#11

SplFileObject is useful when it comes to dealing with large files.

SplFileObject在处理大型文件时非常有用。

function parse_file($filename)
{
    try {
        $file = new SplFileObject($filename);
    } catch (LogicException $exception) {
        die('SplFileObject : '.$exception->getMessage());
    }
    while ($file->valid()) {
        $line = $file->fgets();
        //do something with $line
    }

    //don't forget to free the file handle.
    $file = null;
}

#12

The obvious answer wasn't there in all the responses. PHP has a neat streaming delimiter parser available made for exactly that purpose.

显然答案并不在所有的回复中。PHP有一个简洁的流式分隔符解析器，可以用于此目的。

$fp=fopen()
while ($line = stream_get_line($fp, 1024 * 1024, "\n"))
{
echo $line;
}
fclose($fp);

#13

Function to Read with array return

函数用于读取数组返回值

function read_file($filename = ''){
    $buffer = array();
    $source_file = fopen( $filename, "r" ) or die("Couldn't open $filename");
    while (!feof($source_file)) {
        $buffer[] = fread($source_file, 4096);  // use a buffer of 4KB
    }
    return $buffer;
}