I have a "processor" component that can process a single File, InputStream, Reader, or etc.
我有一个“处理器”组件,可以处理单个文件,InputStream,Reader等。
For various reasons, I end up with several large files instead of one huge file.
由于各种原因,我最终得到了几个大文件而不是一个大文件。
Is there a way to construct an input stream (or reader) that: transparently "appends" all these files so that: 1) The "processor" does not know where one file started or another ended 2) No changes occur in the file system (e.g., no actual appending of files) 3) Each file is read in order so that I do not pay the cost of loading all of them to memory and appending them before the processor starts reading?
有没有办法构建一个输入流(或读取器):透明地“附加”所有这些文件,以便:1)“处理器”不知道一个文件的起始位置或另一个文件的结束2)文件系统中没有发生变化(例如,没有实际附加的文件)3)按顺序读取每个文件,以便我不支付将所有文件加载到内存并在处理器开始读取之前附加它们的成本?
I'm sure it is possible to write something like this, but I'm wondering if one exists already; it's been a while since I did file based IO.
我敢肯定有可能写出这样的东西,但我想知道是否已经存在;自从我做了基于文件的IO以来已经有一段时间了。
2 个解决方案
#1
SequenceInputStream
concatenates multiple streams.
SequenceInputStream连接多个流。
List<InputStream> opened = new ArrayList<InputStream>(files.size());
for (File f : files)
opened.add(new FileInputStream(f));
InputStream is = new SequenceInputStream(Collections.enumeration(opened));
Exception handling (not shown) when opening each file is important; be certain that all files are certain to be closed eventually, even if the operation is aborted before the SequenceInputStream
is created.
打开每个文件时的异常处理(未显示)很重要;确保所有文件最终都肯定会被关闭,即使在创建SequenceInputStream之前中止操作也是如此。
#2
You can use something like SequenceInputStream to read one stream after the other.
您可以使用类似SequenceInputStream的内容来读取另一个流。
#1
SequenceInputStream
concatenates multiple streams.
SequenceInputStream连接多个流。
List<InputStream> opened = new ArrayList<InputStream>(files.size());
for (File f : files)
opened.add(new FileInputStream(f));
InputStream is = new SequenceInputStream(Collections.enumeration(opened));
Exception handling (not shown) when opening each file is important; be certain that all files are certain to be closed eventually, even if the operation is aborted before the SequenceInputStream
is created.
打开每个文件时的异常处理(未显示)很重要;确保所有文件最终都肯定会被关闭,即使在创建SequenceInputStream之前中止操作也是如此。
#2
You can use something like SequenceInputStream to read one stream after the other.
您可以使用类似SequenceInputStream的内容来读取另一个流。