如果/何时取消分配的堆内存被回收？

I have been running overnight memory tests on an embedded Linux system. Using vmstat I have observed that the free memory steadily decreases over time. According to some smaps analysis in procfs, the heap of one process grows at roughly the same rate. I suspected a memory leak and found a few spots in the code where new and delete are regularly used. However, I did not see a new calls without matching delete calls.

我一直在嵌入式Linux系统上运行隔夜内存测试。使用vmstat我发现随着时间的推移,空闲内存会逐渐减少。根据procfs中的一些smaps分析,一个进程的堆以大致相同的速率增长。我怀疑内存泄漏,并在代码中发现了一些经常使用new和delete的地方。但是,我没有看到没有匹配删除调用的新呼叫。

I ran the memory test again and this morning cleared the memory caches with the following call

我再次运行内存测试,今天早上通过以下调用清除了内存缓存

echo 3 > /proc/sys/vm/drop_caches

The free memory listed in vmstat went down to a value close to when the test was started.

vmstat中列出的可用内存下降到接近测试启动时的值。

Does the kernel regularly reclaim unused heap pages? If so, are there other times besides the one above that this is done? Probably when free memory gets below a certain threshold?

内核是否定期回收未使用的堆页面?如果是这样,除了上面的那个之外还有其他时间吗?可能在空闲内存低于某个阈值时?

4 个解决方案

#1

As others said, it is the process's duty to return memory to the kernel.

正如其他人所说,将内存返回内核是进程的责任。

Usually there are 2 ways to allocate memory: if you malloc()/new a memory block above a certain size, the memory gets allocated from the OS via mmap() and eturned as soon as it is free. Smaller blocks are allocated by increasing the process's data area by shifting the sbrk border upwards. This memory is only freed if a block over a certain size is free at the end of that segment.

通常有两种方式来分配内存:如果malloc()/ new一个大小超过一定大小的内存块,内存将通过mmap()从操作系统中分配,并在它空闲时立即进行转换。通过向上移动sbrk边界来增加进程的数据区域来分配较小的块。只有在该段结尾处超过特定大小的块空闲时,才释放该存储器。

E.g.: (pseudo code, I don't know C++ very well)

例如:(伪代码,我不太了解C ++)

a = new char[1000];
b = new char[1000];

Memory map:

---------------+---+---+
end of program | a | b |
---------------+---+---+

If you free a now, you have a hole in the middle. It is not freed because it cannot be freed. If you free b, the process's memory may or may not be reduced; the unused remainder is returned to the system.

如果你现在解放了,你中间就有一个洞。它没有被释放,因为它无法被释放。如果你释放b,过程的记忆可能会或可能不会减少;未使用的余数将返回给系统。

A test with a program as simple as

使用简单的程序进行测试

#include <stdlib.h>

int main()
{
    char * a = malloc(100000);
    char * b = malloc(100000);
    char * c = malloc(100000);
    free(c);
    free(b);
    free(a);
}

leads to a strace output like

导致像strace这样的输出

brk(0)                                  = 0x804b000
brk(0x8084000)                          = 0x8084000
brk(0x80b5000)                          = 0x80b5000
brk(0x809c000)                          = 0x809c000
brk(0x8084000)                          = 0x8084000
brk(0x806c000)                          = 0x806c000

is shows that the brk value is first increased (for malloc()) and then decreased again (for free()).

显示brk值首先增加(对于malloc())然后再次减少(对于free())。

#2

The kernel will reclaim cached memory pages when it needs them, i.e. when the system would otherwise run out of memory. whether memory pages from the processes's heap (free store) are ever returned to the OS is at the discretion of the process's memory manager, in this case the new/delete implementation in the C++ library. This is a completely voluntary operation with which the kernel has nothing to do.

内核将在需要时回收缓存的内存页,即系统否则会耗尽内存。流程的堆(免费存储)中的内存页是否返回到操作系统由进程的内存管理器决定,在这种情况下是C ++库中的新/删除实现。这是一个完全自愿的操作,内核与此无关。

From the fact that drop_caches did the trick, you can infer that it was the kernel cache, not the process's heap, that was filling up memory. Use the free command to find out how much memory is actually available for application use, esp. the -/+ buffers/cache line it reports.

从drop_caches发挥作用的事实来看,你可以推断它是内核缓存,而不是进程的堆,它正在填满内存。使用free命令查看实际可用于应用程序的内存量,尤其是它报告的 - / + buffers / cache行。

#3

Calling delete in your program causes memory to return to the memory manager that is part of your program runtime. In principle this could be written so as to return freed memory to the OS, but I would be surprised if it did. Rather the recycled memory is kept aside for subsequent calls to new.

在程序中调用delete会导致内存返回到程序运行时的内存管理器。原则上,这可以写入以便将释放的内存返回给操作系统,但如果确实如此,我会感到惊讶。而是将回收的内存放在一边,以便随后调用new。

Note that this is the virtual memory of your process; how much of it is actually residing in physical memory at any time during program execution depends on overall system load and is handled by the operating system.

请注意,这是您的流程的虚拟内存;程序执行期间任何时候实际驻留在物理内存中的程度取决于整体系统负载,并由操作系统处理。

#4

User calls to malloc and free (or new and delete), to the best of my knowledge never return no-longer used pages to the O/S. Instead, they just remember what memory has been freed so that if you do a malloc/new of a size that can be satisfied by previously freed memory, then it will use that, rather than going to the O/S and using sbrk to get more memory.

用户调用malloc和free(或new和delete),据我所知,永远不会将不再使用的页面返回给操作系统。相反,他们只记得已经释放了什么内存,这样如果你做了一个大小可以通过以前释放的内存来满足的malloc / new,那么它将使用它,而不是去O / S并使用sbrk来获取更多的记忆。

Thus this code:

因此这段代码:

for (;;)
{
    struct { char data[200 * 1024 * 1024] } HugeBuffer;
    HugeBuffer *buff = new HugeBuffer;
    delete buff;
}

Will allocate 200Mb once, and then just steadily use that memory forever. It will go to the O/S once on the original allocation, and then loop fiddling around in user space.

将分配200Mb一次,然后永远稳定地使用该内存。它将在原始分配上进入O / S一次,然后在用户空间中进行循环。

#1