FileInputStream RandomAccessFile BufferedReader MappedByteBuffer 大文件数据读取效率测试

时间:2022-08-06 19:31:22
文件大小为67M:
public class App {
String generalFile = "/opt/A";

public void generalFileRead(String fileName) {
try {
System.out.print("readGeneralFile");
FileInputStream in = new FileInputStream(fileName);
readIn(in);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}

}

public void randomFileRead(String fileName) {
try {
System.out.print("readRandomFile");
RandomAccessFile file = new RandomAccessFile(fileName, "r");
int i = file.read();
while (i != -1) {
i = file.read();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

public void bufferRead(String fileName) {
try {
System.out.print("readBufferInput");
InputStream in = new BufferedInputStream(new FileInputStream(
fileName));
BufferedReader bf = new BufferedReader(new InputStreamReader(in));
while(bf.readLine()!=null){
bf.readLine();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

private void readIn(InputStream in) throws IOException {
int i = in.read();
while (i != -1) {
i = in.read();
}
}

public void mappedByteBufferRead(String fileName) {
try {
System.out.print("readMappedFile");
FileInputStream in = new FileInputStream(fileName);

FileChannel channel = in.getChannel();
int length = (int) channel.size();
MappedByteBuffer buffer = channel.map(
FileChannel.MapMode.READ_ONLY, 0, length);
while(buffer.remaining()>0){
buffer.get();
}

} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

public void countTime(long start) {
System.out.println(":" + (System.currentTimeMillis() - start) + "ms");
}

public static void main(String[] args) {
App app = new App();

long start = System.currentTimeMillis();
app.generalFileRead(app.generalFile);
app.countTime(start);

start = System.currentTimeMillis();
app.randomFileRead(app.generalFile);
app.countTime(start);

start = System.currentTimeMillis();
app.bufferRead(app.generalFile);
app.countTime(start);

start = System.currentTimeMillis();
app.mappedByteBufferRead(app.generalFile);
app.countTime(start);

}

}

readGeneralFile:16981ms

readRandomFile:16868ms

readBufferInput:305ms

readMappedFile:11ms

MappedByteBuffer胜!

另外测试了6G大小的文件(只测试了BufferReader和MappedByteBuffer),BufferReader 可以顺利读下来,MappedByteBuffer 文件size()大小超出了Integer的最大值,需要分段读取。

做了按行读取测试,结果是 BufferReader 快于 MappedByteBuffer,应该是BufferReader.readLine() 优化过!但MappedByteBuffer 没有readLine()函数,自己判断10和13实现,水平有限效率不高。比较下来MappedByteBuffer 慢。

因为工作中用到读取行,只做了MappedByteBuffer和BufferReader的读取行的测试,没有测试比较读取字节的性能。