Is there a maximum number of inodes in a single directory?
在一个目录中是否存在最大的索引节点数?
I have a directory of over 2 million files and can't get the ls command to work against that directory. So now I'm wondering if I've exceeded a limit on inodes in Linux. Is there a limit before a 2^64 numerical limit?
我有一个超过200万个文件的目录，无法让ls命令在该目录下工作。所以现在我想知道我是否已经超过了Linux中inode的限制。有限制在2 ^ 64数值限制?
10 个解决方案
#1
50
df -i should tell you the number of inodes used and free on the file system.
df -我应该告诉您在文件系统中使用和免费的inode的数量。
#2
17
Try ls -U or ls -f.
试试ls -U或ls -f。
ls, by default, sorts the files alphabetically. If you have 2 million files, that sort can take a long time. If ls -U (or perhaps ls -f), then the file names will be printed immediately.
默认情况下，ls按字母顺序对文件进行排序。如果您有200万个文件，那么这类文件可能需要很长时间。如果ls -U(或者ls -f)，则立即打印文件名。
#3
10
No. Inode limits are per-filesystem, and decided at filesystem creation time. You could be hitting another limit, or maybe 'ls' just doesn't perform that well.
不。Inode限制是每个文件系统，并决定在文件系统创建时间。你可能会达到另一个极限，或者“ls”表现不太好。
Try this:
试试这个:
tune2fs -l /dev/DEVICE | grep -i inode
It should tell you all sorts of inode related info.
它应该告诉你所有的inode相关信息。
#4
4
What you hit is an internal limit of ls. Here is an article which explains it quite well: http://www.olark.com/spw/2011/08/you-can-list-a-directory-with-8-million-files-but-not-with-ls/
你击中的是ls的内部极限。这里有一篇文章很好地解释了这一点:http://www.olark.com/spw/2011/08/can -list-a-directory-with- 800万files-but-not-with-ls/
#5
3
Maximum directory size is filesystem-dependent, and thus the exact limit varies. However, having very large directories is a bad practice.
最大目录大小依赖于文件系统，因此确切的限制是不同的。然而，拥有非常大的目录是一种不好的做法。
You should consider making your directories smaller by sorting files into subdirectories. One common scheme is to use the first two characters for a first-level subdirectory, as follows:
您应该考虑将文件排序到子目录中，从而使目录更小。一种常见的方案是使用第一级子目录的前两个字符，如下所示:
${topdir}/aa/aardvark
${topdir}/ai/airplane
This works particularly well if using UUID, GUIDs or content hash values for naming.
如果使用UUID、GUIDs或内容哈希值进行命名，这将非常有效。
#6
1
As noted by Rob Adams, ls is sorting the files before displaying them. Note that if you are using NFS, the NFS server will be sorting the directory before sending it, and 2 million entries may well take longer than the NFS timeout. That makes the directory unlistable via NFS, even with the -f flag.
正如Rob Adams所指出的，ls是在显示文件之前对其进行排序。请注意，如果使用NFS, NFS服务器将在发送之前对目录进行排序，而200万条条目可能比NFS超时花费的时间更长。这使得该目录通过NFS不可列表，即使使用-f标志。
This may be true for other network file systems as well.
这对其他网络文件系统也是一样的。
While there's no enforced limit to the number of entries in a directory, it's good practice to have some limit to the entries you anticipate.
虽然对目录中的条目数量没有强制限制，但是对您预期的条目设置一些限制是很好的实践。
#7
0
Can you get a real count of the number of files? Does it fall very near a 2^n boundry? Could you simply be running out of RAM to hold all the file names?
你能算出文件的数量吗?它下降很2 ^ n边界附近吗?你能不能用完内存来保存所有的文件名?
I know that in windows at least file system performance would drop dramatically as the number of files in the folder went up, but I thought that linux didn't suffer from this issue, at least if you were using a command prompt. God help you if you try to get something like nautilus to open a folder with that many files.
我知道，至少在windows中，文件系统性能会随着文件夹中文件数量的增加而急剧下降，但我认为linux不会受到这个问题的影响，至少如果您使用命令提示符的话。如果你想让诺第留斯号这样的东西打开一个有这么多文件的文件夹，上帝会帮助你的。
I'm also wondering where these files come from. Are you able to calculate file names programmatically? If that's the case, you might be able to write a small program to sort them into a number of sub-folders. Often listing the name of a specific file will grant you access where trying to look up the name will fail. For example, I have a folder in windows with about 85,000 files where this works.
我也想知道这些文件是从哪里来的。你能以编程方式计算文件名吗?如果是这样的话，您可能可以编写一个小程序将它们分类到多个子文件夹中。通常列出特定文件的名称将允许您访问试图查找该名称将失败的地方。例如，我在windows中有一个文件夹，其中大约有85,000个文件可以使用。
If this technique is successful, you might try finding a way to make this sort permanent, even if it's just running this small program as a cron job. It'll work especially well if you can sort the files by date somewhere.
如果这种技术成功，您可以尝试寻找一种方法使这种类型永久存在，即使它只是作为cron作业运行这个小程序。如果你能按日期对这些文件进行分类，它将会非常有效。
#8
0
Unless you are getting an error message, ls is working but very slowly. You can try looking at just the first ten files like this:
除非您得到一个错误消息，否则ls的工作非常缓慢。你可以试着看看前十个这样的文件:
ls -f | head -10
l -f |头-10。
If you're going to need to look at the file details for a while, you can put them in a file first. You probably want to send the output to a different directory than the one you are listing at the moment!
如果您需要查看一段时间的文件细节，您可以先将它们放在一个文件中。您可能希望将输出发送到与当前列出的目录不同的目录!
ls > ~/lots-of-files.txt
ls > ~ / lots-of-files.txt
If you want to do something to the files, you can use xargs. If you decide to write a script of some kind to do the work, make sure that your script will process the list of files as a stream rather than all at once. Here's an example of moving all the files.
如果你想对文件做些什么，你可以使用xargs。如果您决定编写某种脚本来完成这项工作，请确保您的脚本将把文件列表作为流处理，而不是一次性处理所有文件。这里有一个移动所有文件的例子。
ls | xargs -I thefilename mv thefilename ~/some/other/directory
l | xargs -我的文件名mv文件名~/some/other/directory
You could combine that with head to move a smaller number of the files.
您可以将其与head合并，以移动较小数量的文件。
ls | head -10000 | xargs -I x mv x /first/ten/thousand/files/go/here
| head -10000 | xargs -I x mv x /first/ 10 / 1000 /文件/go/这里。
You can probably combine ls | head into a shell script to that will split up the files into a bunch of directories with a manageable number of files in each.
您可能可以将ls | head合并到一个shell脚本中，该脚本将文件分割为多个目录，每个目录中都有可管理的文件。
#9
0
For NetBackup, the binaries that analyze the directories in clients perform some type of listing that timeouts by the enormous quantity of files in every folder (about one million per folder, SAP work directory).
对于NetBackup，分析客户端目录的二进制文件执行某种类型的列表，该列表按每个文件夹中的大量文件(每个文件夹大约100万个，SAP工作目录)超时。
My solution was (as Charles Duffy write in this thread), reorganize the folders in subfolders with less archives.
我的解决方案是(就像Charles Duffy在这个帖子里写的那样)，用更少的档案重新整理子文件夹中的文件夹。
#10
-1
Another option is find:
另一个选择是找到:
find . -name * -exec somcommands {} \;
找到。-name * -exec somcommands {};
{} is the absolute filepath.
{}是绝对文件路径。
The advantage/disadvantage is that the files are processed one after each other.
优点/缺点是文件被一个接一个地处理。
find . -name * > ls.txt
找到。- name * > ls.txt
would print all filenames in ls.txt
会打印所有的文件名在ls.txt
find . -name * -exec ls -l {} \; > ls.txt
找到。-name * -exec ls -l {} \;> ls.txt
would print all information form ls for each file in ls.txt
是否要在ls.txt中打印每个文件的所有信息表单ls
#1
50
df -i should tell you the number of inodes used and free on the file system.
df -我应该告诉您在文件系统中使用和免费的inode的数量。
#2
17
Try ls -U or ls -f.
试试ls -U或ls -f。
ls, by default, sorts the files alphabetically. If you have 2 million files, that sort can take a long time. If ls -U (or perhaps ls -f), then the file names will be printed immediately.
默认情况下，ls按字母顺序对文件进行排序。如果您有200万个文件，那么这类文件可能需要很长时间。如果ls -U(或者ls -f)，则立即打印文件名。
#3
10
No. Inode limits are per-filesystem, and decided at filesystem creation time. You could be hitting another limit, or maybe 'ls' just doesn't perform that well.
不。Inode限制是每个文件系统，并决定在文件系统创建时间。你可能会达到另一个极限，或者“ls”表现不太好。
Try this:
试试这个:
tune2fs -l /dev/DEVICE | grep -i inode
It should tell you all sorts of inode related info.
它应该告诉你所有的inode相关信息。
#4
4
What you hit is an internal limit of ls. Here is an article which explains it quite well: http://www.olark.com/spw/2011/08/you-can-list-a-directory-with-8-million-files-but-not-with-ls/
你击中的是ls的内部极限。这里有一篇文章很好地解释了这一点:http://www.olark.com/spw/2011/08/can -list-a-directory-with- 800万files-but-not-with-ls/
#5
3
Maximum directory size is filesystem-dependent, and thus the exact limit varies. However, having very large directories is a bad practice.
最大目录大小依赖于文件系统，因此确切的限制是不同的。然而，拥有非常大的目录是一种不好的做法。
You should consider making your directories smaller by sorting files into subdirectories. One common scheme is to use the first two characters for a first-level subdirectory, as follows:
您应该考虑将文件排序到子目录中，从而使目录更小。一种常见的方案是使用第一级子目录的前两个字符，如下所示:
${topdir}/aa/aardvark
${topdir}/ai/airplane
This works particularly well if using UUID, GUIDs or content hash values for naming.
如果使用UUID、GUIDs或内容哈希值进行命名，这将非常有效。
#6
1
As noted by Rob Adams, ls is sorting the files before displaying them. Note that if you are using NFS, the NFS server will be sorting the directory before sending it, and 2 million entries may well take longer than the NFS timeout. That makes the directory unlistable via NFS, even with the -f flag.
正如Rob Adams所指出的，ls是在显示文件之前对其进行排序。请注意，如果使用NFS, NFS服务器将在发送之前对目录进行排序，而200万条条目可能比NFS超时花费的时间更长。这使得该目录通过NFS不可列表，即使使用-f标志。
This may be true for other network file systems as well.
这对其他网络文件系统也是一样的。
While there's no enforced limit to the number of entries in a directory, it's good practice to have some limit to the entries you anticipate.
虽然对目录中的条目数量没有强制限制，但是对您预期的条目设置一些限制是很好的实践。
#7
0
Can you get a real count of the number of files? Does it fall very near a 2^n boundry? Could you simply be running out of RAM to hold all the file names?
你能算出文件的数量吗?它下降很2 ^ n边界附近吗?你能不能用完内存来保存所有的文件名?
I know that in windows at least file system performance would drop dramatically as the number of files in the folder went up, but I thought that linux didn't suffer from this issue, at least if you were using a command prompt. God help you if you try to get something like nautilus to open a folder with that many files.
我知道，至少在windows中，文件系统性能会随着文件夹中文件数量的增加而急剧下降，但我认为linux不会受到这个问题的影响，至少如果您使用命令提示符的话。如果你想让诺第留斯号这样的东西打开一个有这么多文件的文件夹，上帝会帮助你的。
I'm also wondering where these files come from. Are you able to calculate file names programmatically? If that's the case, you might be able to write a small program to sort them into a number of sub-folders. Often listing the name of a specific file will grant you access where trying to look up the name will fail. For example, I have a folder in windows with about 85,000 files where this works.
我也想知道这些文件是从哪里来的。你能以编程方式计算文件名吗?如果是这样的话，您可能可以编写一个小程序将它们分类到多个子文件夹中。通常列出特定文件的名称将允许您访问试图查找该名称将失败的地方。例如，我在windows中有一个文件夹，其中大约有85,000个文件可以使用。
If this technique is successful, you might try finding a way to make this sort permanent, even if it's just running this small program as a cron job. It'll work especially well if you can sort the files by date somewhere.
如果这种技术成功，您可以尝试寻找一种方法使这种类型永久存在，即使它只是作为cron作业运行这个小程序。如果你能按日期对这些文件进行分类，它将会非常有效。
#8
0
Unless you are getting an error message, ls is working but very slowly. You can try looking at just the first ten files like this:
除非您得到一个错误消息，否则ls的工作非常缓慢。你可以试着看看前十个这样的文件:
ls -f | head -10
l -f |头-10。
If you're going to need to look at the file details for a while, you can put them in a file first. You probably want to send the output to a different directory than the one you are listing at the moment!
如果您需要查看一段时间的文件细节，您可以先将它们放在一个文件中。您可能希望将输出发送到与当前列出的目录不同的目录!
ls > ~/lots-of-files.txt
ls > ~ / lots-of-files.txt
If you want to do something to the files, you can use xargs. If you decide to write a script of some kind to do the work, make sure that your script will process the list of files as a stream rather than all at once. Here's an example of moving all the files.
如果你想对文件做些什么，你可以使用xargs。如果您决定编写某种脚本来完成这项工作，请确保您的脚本将把文件列表作为流处理，而不是一次性处理所有文件。这里有一个移动所有文件的例子。
ls | xargs -I thefilename mv thefilename ~/some/other/directory
l | xargs -我的文件名mv文件名~/some/other/directory
You could combine that with head to move a smaller number of the files.
您可以将其与head合并，以移动较小数量的文件。
ls | head -10000 | xargs -I x mv x /first/ten/thousand/files/go/here
| head -10000 | xargs -I x mv x /first/ 10 / 1000 /文件/go/这里。
You can probably combine ls | head into a shell script to that will split up the files into a bunch of directories with a manageable number of files in each.
您可能可以将ls | head合并到一个shell脚本中，该脚本将文件分割为多个目录，每个目录中都有可管理的文件。
#9
0
For NetBackup, the binaries that analyze the directories in clients perform some type of listing that timeouts by the enormous quantity of files in every folder (about one million per folder, SAP work directory).
对于NetBackup，分析客户端目录的二进制文件执行某种类型的列表，该列表按每个文件夹中的大量文件(每个文件夹大约100万个，SAP工作目录)超时。
My solution was (as Charles Duffy write in this thread), reorganize the folders in subfolders with less archives.
我的解决方案是(就像Charles Duffy在这个帖子里写的那样)，用更少的档案重新整理子文件夹中的文件夹。
#10
-1
Another option is find:
另一个选择是找到:
find . -name * -exec somcommands {} \;
找到。-name * -exec somcommands {};
{} is the absolute filepath.
{}是绝对文件路径。
The advantage/disadvantage is that the files are processed one after each other.
优点/缺点是文件被一个接一个地处理。
find . -name * > ls.txt
找到。- name * > ls.txt
would print all filenames in ls.txt
会打印所有的文件名在ls.txt
find . -name * -exec ls -l {} \; > ls.txt
找到。-name * -exec ls -l {} \;> ls.txt
would print all information form ls for each file in ls.txt
是否要在ls.txt中打印每个文件的所有信息表单ls