There are several high quality frameworks that hide the complexity of NIO based network programming (mina, netty, grizzly, etc.). Are there similar frameworks that simplify NIO based file-system programming?
有几个高质量的框架可以隐藏基于NIO的网络编程(mina,netty,grizzly等)的复杂性。是否有类似的框架可以简化基于NIO的文件系统编程?
For example, as a learning exercise, I would like to implement a disk backed Map based on this (awesome!) article: http://www.javaworld.com/javaworld/jw-01-1999/jw-01-step.html.
例如,作为一个学习练习,我想基于这篇(真棒!)文章实现一个磁盘支持的Map:http://www.javaworld.com/javaworld/jw-01-1999/jw-01-step。 HTML。
2 个解决方案
#1
No (but...)
But that is because Java's NIO FileChannel and MappedByteBuffer are not nearly as complex or difficult to understand and use as the networking and Selector stuff java.nio.
但这是因为Java的NIO FileChannel和MappedByteBuffer并不像网络和Selector那样复杂或难以理解和使用java.nio。
Here is an example of creating a disk backed map (known as a 'mapped byte buffer' in NIO-land) that would be appropriate for your exercise:
下面是一个创建磁盘支持映射(在NIO-land中称为“映射字节缓冲区”)的示例,该映射适用于您的练习:
File file = new File("/Users/stu/mybigfile.bin");
FileChannel fc = (new FileInputStream(file)).getChannel();
MappedByteBuffer buf = fc.map(MapMode.READ_WRITE, 0, file.length());
You can access the buffer like any other Buffer. Data moves magically and quickly between disk and memory, all managed by Java and the underlying OS's virtual memory management system. You do have a degree of control of this, though. E.g.: MappedByteBuffer's .force() ('Forces any changes made to this buffer's content to be written to the storage device containing the mapped file.') and .load() ('Loads this buffer's content into physical memory.') I've never needed these personally.
您可以像任何其他缓冲区一样访问缓冲区。数据在磁盘和内存之间神奇而快速地移动,全部由Java和底层操作系统的虚拟内存管理系统管理。不过,你确实对此有一定程度的控制。例如:MappedByteBuffer的.force()('强制对此缓冲区的内容所做的任何更改将写入包含映射文件的存储设备。')和.load()('将此缓冲区的内容加载到物理内存中'。)I'我个人从不需要这些。
#2
To add to @Stu's comment. It worth noting that socket connections do not have all their data at once but instead may need to support many slow connections (esp connections which are open but no data is sent yet)
添加到@Stu的评论。值得注意的是,套接字连接不会同时拥有所有数据,而是可能需要支持许多慢速连接(esp连接已打开但尚未发送数据)
However for files, all the data is available at once and you typically only need to open a few files at a time to get maximum performance (often one at a time is fine) If you are loading data from multiple drives (rare) or from multiple servers (very rare) or multiple network interfaces (even rarer) you might fine accessing a few files at a time improves performance. Even then the complexity is not high and you can just create a thread for each file you are loading.
但是对于文件,所有数据都是一次可用的,您通常只需要一次打开几个文件以获得最大性能(通常一次只能一个)如果要从多个驱动器(罕见)或从中加载数据多个服务器(非常罕见)或多个网络接口(甚至更少)您可以一次访问几个文件,从而提高性能。即使这样,复杂性也不高,您只需为要加载的每个文件创建一个线程。
The only occasion where files are complicated is reading log files. This complicated as the file can grow in size as you read it. You can reach the end of the files and later find more data. Also log files can be rotated meaning the file you had open is no longer the file you want. Even so this is not very difficult to deal with and a fairly rare requirement.
文件复杂的唯一场合是读取日志文件。这很复杂,因为文件在读取时会变大。您可以到达文件的末尾,然后查找更多数据。此外,日志文件可以旋转,这意味着您打开的文件不再是您想要的文件。即便如此,这也不是很难处理,也是一个相当罕见的要求。
#1
No (but...)
But that is because Java's NIO FileChannel and MappedByteBuffer are not nearly as complex or difficult to understand and use as the networking and Selector stuff java.nio.
但这是因为Java的NIO FileChannel和MappedByteBuffer并不像网络和Selector那样复杂或难以理解和使用java.nio。
Here is an example of creating a disk backed map (known as a 'mapped byte buffer' in NIO-land) that would be appropriate for your exercise:
下面是一个创建磁盘支持映射(在NIO-land中称为“映射字节缓冲区”)的示例,该映射适用于您的练习:
File file = new File("/Users/stu/mybigfile.bin");
FileChannel fc = (new FileInputStream(file)).getChannel();
MappedByteBuffer buf = fc.map(MapMode.READ_WRITE, 0, file.length());
You can access the buffer like any other Buffer. Data moves magically and quickly between disk and memory, all managed by Java and the underlying OS's virtual memory management system. You do have a degree of control of this, though. E.g.: MappedByteBuffer's .force() ('Forces any changes made to this buffer's content to be written to the storage device containing the mapped file.') and .load() ('Loads this buffer's content into physical memory.') I've never needed these personally.
您可以像任何其他缓冲区一样访问缓冲区。数据在磁盘和内存之间神奇而快速地移动,全部由Java和底层操作系统的虚拟内存管理系统管理。不过,你确实对此有一定程度的控制。例如:MappedByteBuffer的.force()('强制对此缓冲区的内容所做的任何更改将写入包含映射文件的存储设备。')和.load()('将此缓冲区的内容加载到物理内存中'。)I'我个人从不需要这些。
#2
To add to @Stu's comment. It worth noting that socket connections do not have all their data at once but instead may need to support many slow connections (esp connections which are open but no data is sent yet)
添加到@Stu的评论。值得注意的是,套接字连接不会同时拥有所有数据,而是可能需要支持许多慢速连接(esp连接已打开但尚未发送数据)
However for files, all the data is available at once and you typically only need to open a few files at a time to get maximum performance (often one at a time is fine) If you are loading data from multiple drives (rare) or from multiple servers (very rare) or multiple network interfaces (even rarer) you might fine accessing a few files at a time improves performance. Even then the complexity is not high and you can just create a thread for each file you are loading.
但是对于文件,所有数据都是一次可用的,您通常只需要一次打开几个文件以获得最大性能(通常一次只能一个)如果要从多个驱动器(罕见)或从中加载数据多个服务器(非常罕见)或多个网络接口(甚至更少)您可以一次访问几个文件,从而提高性能。即使这样,复杂性也不高,您只需为要加载的每个文件创建一个线程。
The only occasion where files are complicated is reading log files. This complicated as the file can grow in size as you read it. You can reach the end of the files and later find more data. Also log files can be rotated meaning the file you had open is no longer the file you want. Even so this is not very difficult to deal with and a fairly rare requirement.
文件复杂的唯一场合是读取日志文件。这很复杂,因为文件在读取时会变大。您可以到达文件的末尾,然后查找更多数据。此外,日志文件可以旋转,这意味着您打开的文件不再是您想要的文件。即便如此,这也不是很难处理,也是一个相当罕见的要求。