驻留集大小(RSS)限制无效

时间:2022-08-02 18:11:43

The following problem occurs on a machine running Ubuntu 10.04 with the 2.6.32-22-generic kernel: Setting a limit for the Resident Set Size (RSS) of a process does not seem to have any effect. I currently set the limit in Python with the following code:

在运行带有2.6.32-22通用内核的Ubuntu 10.04的计算机上会出现以下问题:设置进程的驻留集大小(RSS)的限制似乎没有任何影响。我目前使用以下代码在Python中设置限制:

import resource
# (100, 100) is the (soft, hard) limit. ~100kb.
resource.setrlimit(resource.RLIMIT_RSS, (100, 100))
memory_sink = ['a']*10000000   # this should fail

The list, memory_sink, succeeds every time. When I check RSS usage with top, I can easily get the process to use 1gb of RAM, which means that the limit is not working. Do RSS limits not work with this kernel or distro? If it helps, resource.RLIMIT_NPROC (user process limit) does work.

列表memory_sink每次都成功。当我使用top检查RSS使用情况时,我可以很容易地使用1gb的RAM,这意味着限制不起作用。 RSS限制不适用于此内核或发行版吗?如果有帮助,resource.RLIMIT_NPROC(用户进程限制)确实有效。

4 个解决方案

#1


12  

Form the getrlimit manpage:

形成getrlimit联机帮助页:

RLIMIT_RSS
Specifies the limit (in pages) of  the  process's  resident  set
(the  number of virtual pages resident in RAM).  This limit only
has effect in Linux 2.4.x, x < 30, and there only affects  calls
to madvise(2) specifying MADV_WILLNEED.

It seems this is just not supported on Linux kernel 2.6.

看起来这在Linux内核2.6上是不受支持的。

#2


19  

You can accomplish this using cgroups. The long version is on my blog, but the short version (tested on Ubuntu 11.04) is:

您可以使用cgroups完成此操作。长版本在我的博客上,但是短版本(在Ubuntu 11.04上测试)是:

  • Install the cgroup-bin package.

    安装cgroup-bin包。

  • Edit /etc/cgconfig.config and create a group with limited memory. For instance, I added:

    编辑/etc/cgconfig.config并创建内存有限的组。例如,我补充说:

    group limited {
      memory {
        memory.limit_in_bytes = 50M;
      }
    }
    
  • Run

    $ sudo restart cgconfig
    $ sudo chown -R jlebar /sys/fs/cgroup/memory/limited
    $ cgexec -g memory:limited your/program
    

I observed my process with an RSS of 93M when I asked it to use only 50M, but that wasn't a problem for me, since my goal was just to get the program to page.

当我要求它只使用50M时,我用93M的RSS观察了我的过程,但这对我来说不是问题,因为我的目标只是让程序进入页面。

cgclassify lets you attach restrictions to a running process too. Note for RSS this only applies to memory allocated after the restriction comes into effect.

cgclassify还允许您对正在运行的进程附加限制。 RSS注意这仅适用于限制生效后分配的内存。

#3


3  

A related limit - virtual memory or address space (RLIMIT_AS) - does work. This allows limiting the python process and subprocesses memory without external tools.

相关的限制 - 虚拟内存或地址空间(RLIMIT_AS) - 确实有效。这允许在没有外部工具的情况下限制python进程和子进程内存。

>>> size = 50*1024*1024 # In bytes
>>> resource.setrlimit(resource.RLIMIT_AS, (size, resource.RLIM_INFINITY))
>>> a = 'a' * size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError

From the man page:

从手册页:

RLIMIT_AS. The maximum size of the process's virtual memory (address space) in bytes.

RLIMIT_AS。进程的虚拟内存(地址空间)的最大大小(以字节为单位)。

Here is a good explanation of the difference between the Resident Set and the VM size - What is RSS and VSZ in Linux memory management.

以下是对Resident Set和VM大小之间差异的一个很好的解释 - Linux内存管理中的RSS和VSZ是什么。

#4


0  

I created a script to limit memory usage using cgroups and cgroup manager, useable for ad-hoc commands and not needing root privileges. See https://unix.stackexchange.com/questions/134414/how-to-limit-the-total-resources-memory-of-a-process-and-its-children/174894#174894

我使用cgroups和cgroup manager创建了一个限制内存使用的脚本,可用于ad-hoc命令而不需要root权限。请参阅https://unix.stackexchange.com/questions/134414/how-to-limit-the-total-resources-memory-of-a-process-and-its-children/174894#174894

#1


12  

Form the getrlimit manpage:

形成getrlimit联机帮助页:

RLIMIT_RSS
Specifies the limit (in pages) of  the  process's  resident  set
(the  number of virtual pages resident in RAM).  This limit only
has effect in Linux 2.4.x, x < 30, and there only affects  calls
to madvise(2) specifying MADV_WILLNEED.

It seems this is just not supported on Linux kernel 2.6.

看起来这在Linux内核2.6上是不受支持的。

#2


19  

You can accomplish this using cgroups. The long version is on my blog, but the short version (tested on Ubuntu 11.04) is:

您可以使用cgroups完成此操作。长版本在我的博客上,但是短版本(在Ubuntu 11.04上测试)是:

  • Install the cgroup-bin package.

    安装cgroup-bin包。

  • Edit /etc/cgconfig.config and create a group with limited memory. For instance, I added:

    编辑/etc/cgconfig.config并创建内存有限的组。例如,我补充说:

    group limited {
      memory {
        memory.limit_in_bytes = 50M;
      }
    }
    
  • Run

    $ sudo restart cgconfig
    $ sudo chown -R jlebar /sys/fs/cgroup/memory/limited
    $ cgexec -g memory:limited your/program
    

I observed my process with an RSS of 93M when I asked it to use only 50M, but that wasn't a problem for me, since my goal was just to get the program to page.

当我要求它只使用50M时,我用93M的RSS观察了我的过程,但这对我来说不是问题,因为我的目标只是让程序进入页面。

cgclassify lets you attach restrictions to a running process too. Note for RSS this only applies to memory allocated after the restriction comes into effect.

cgclassify还允许您对正在运行的进程附加限制。 RSS注意这仅适用于限制生效后分配的内存。

#3


3  

A related limit - virtual memory or address space (RLIMIT_AS) - does work. This allows limiting the python process and subprocesses memory without external tools.

相关的限制 - 虚拟内存或地址空间(RLIMIT_AS) - 确实有效。这允许在没有外部工具的情况下限制python进程和子进程内存。

>>> size = 50*1024*1024 # In bytes
>>> resource.setrlimit(resource.RLIMIT_AS, (size, resource.RLIM_INFINITY))
>>> a = 'a' * size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError

From the man page:

从手册页:

RLIMIT_AS. The maximum size of the process's virtual memory (address space) in bytes.

RLIMIT_AS。进程的虚拟内存(地址空间)的最大大小(以字节为单位)。

Here is a good explanation of the difference between the Resident Set and the VM size - What is RSS and VSZ in Linux memory management.

以下是对Resident Set和VM大小之间差异的一个很好的解释 - Linux内存管理中的RSS和VSZ是什么。

#4


0  

I created a script to limit memory usage using cgroups and cgroup manager, useable for ad-hoc commands and not needing root privileges. See https://unix.stackexchange.com/questions/134414/how-to-limit-the-total-resources-memory-of-a-process-and-its-children/174894#174894

我使用cgroups和cgroup manager创建了一个限制内存使用的脚本,可用于ad-hoc命令而不需要root权限。请参阅https://unix.stackexchange.com/questions/134414/how-to-limit-the-total-resources-memory-of-a-process-and-its-children/174894#174894