快速RAM中的动态内存分配

On a Windows 32-bit and 64-bit machine, I have to allocate memory to store large amounts of data that are streaming live, a total of around 1GB. If I use malloc(), I am going to obtain a virtual memory address, and this address could be actually causing some paging to the hard drive depending on the amount of memory I have. Unfortunately I'm afraid that the HD will impact performance and cause data to be missing.

在32位和64位的Windows机器上，我必须分配内存来存储大量的实时数据，总共大约1GB。如果我使用malloc()，我将获得一个虚拟内存地址，这个地址实际上可能会导致对硬盘驱动器的一些分页，这取决于我有多少内存。不幸的是，我担心HD会影响性能并导致数据丢失。

Is there a way to force memory to allocate only in RAM, even if it means that I get an error when not enough memory is available (so the user needs to close other things or use another machine)? I want to guarantee that all operations will be done in memory. If this fails, forcing the application to exit is acceptable.

是否有一种方法可以强制内存只在RAM中进行分配，即使这意味着在没有足够内存的情况下(因此用户需要关闭其他东西或使用另一台机器)，我也会出现错误?我想保证所有操作都将在内存中完成。如果失败，则强制应用程序退出是可以接受的。

I know that another process may come in and itself take some memory, but I am not worried because in this machine that is not happening (it'll be the only application on the machine to be doing this large allocation).

我知道另一个进程可能会进来并占用一些内存，但我并不担心，因为在这台机器中没有发生这种情况(这将是机器上唯一一个执行这种大分配的应用程序)。

[Edit:] My attempt so far has been to try use VirtualLock as follows:

[编辑:]到目前为止，我一直在尝试使用VirtualLock，如下所示:

if(!SetProcessWorkingSetSize(this, 300000, 300008))
    printf("Error Changing Working Set Size\n");

// Allocate 1GB space
unsigned long sz = sizeof(unsigned char)*1000000000;
unsigned char * m_buffer = (unsigned char *) malloc(sz);

if(m_buffer == NULL)
{
    printf("Memory Allocation failed\n");
}
else
{
    // Protect memory from being swapped
    if(!VirtualLock(m_buffer , sz))
    {
           printf("Memory swap protection failed\n");
    }           
}

But the change in Working set fails, and so does the VirtualLock. Malloc does return non-null.

但是工作集中的更改失败了，虚拟化锁也失败了。Malloc返回null。

[Edit2] I have tried also:

[Edit2]我也试过:

 unsigned long sz = sizeof(unsigned char)*1000000000;
 LPVOID lpvResult;
 lpvResult = VirtualAlloc(NULL,sz, MEM_PHYSICAL|MEM_RESERVE, PAGE_NOCACHE);

But lpvResult is 0, so no luck there either.

但是lpvResult是0，所以这里也没有运气。

5 个解决方案

#1

You can use mlock, mlockall, munlock, munlockall functions in order to prevent pages from being swapped (part of POSIX, also available in MinGW). Unfortunately, I have no experience with Windows but it looks like VirtualLock does the same thing.

您可以使用mlock、mlockall、munlock、munlockall函数来防止页面交换(POSIX的一部分，也可以在MinGW中使用)。不幸的是，我没有使用Windows的经验，但是看起来VirtualLock做了同样的事情。

Hope it helps. Good Luck!

希望它可以帮助。好运！

#2

I think VirtualAlloc might get you some of what you want.

我认为虚拟人可能会给你带来一些你想要的东西。

This problem really boils down to just writing your own memory manager instead of using CRT function.

这个问题可以归结为编写自己的内存管理器，而不是使用CRT函数。

#3

You need to use the undocumented NtLockVirtualMemory function with lock option 2 (LOCK_VM_IN_RAM); make sure you request and obtain SE_LOCK_MEMORY_NAME privilege first, and be aware that it might not be granted (I'm sure what the group policy defaults the privilege to, but it might very well be granted to nobody).

您需要使用无文档的NtLockVirtualMemory函数和锁选项2 (LOCK_VM_IN_RAM);请确保您首先请求并获取SE_LOCK_MEMORY_NAME特权，并注意它可能不会被授予(我确定组策略默认授予的特权是什么，但是很可能不会授予任何人)。

I suggest using VirtualLock as a fallback, and if that fails too, to use SetProcessWorkingSetSize. If that fails then just let it fail I guess...

我建议使用VirtualLock作为回退，如果失败，也可以使用SetProcessWorkingSetSize。如果失败了，那就让它失败吧，我猜……

See this link for some nice discussion about this. One person says:

有关这方面的一些精彩讨论，请参见这个链接。一个人说:

When you specify LOCK_VM_IN_WSL flag, you just tell the Balance Set Manager that you don't want some particular page to get swapped to the disk, and ask it to leave this page alone when trimming the working set of the target process. This is just an indication, so that the target page may still get swapped if the system is low on RAM. However, when you specify LOCK_VM_IN_RAM flag, you issue a directive to the Memory Manager to treat this page as non-pageable (i.e. do something the driver does when it calls MmProbeAndLockPages() in order to lock pages, described by MDL) , so that the page is question is guaranteed to be loaded in RAM all the time.

当您指定LOCK_VM_IN_WSL标志时，您只需告诉Balance Set Manager，您不希望某些特定的页面被交换到磁盘上，并要求它在修剪目标进程的工作集时不要占用这个页面。这只是一个指示，因此如果系统在RAM中处于低位，那么目标页面可能仍然会被交换。然而,当您指定LOCK_VM_IN_RAM国旗,你发出指令的内存管理器将这个页面作为non-pageable(即做司机当它调用MmProbeAndLockPages()为了锁定页面,描述了MDL),这样页面问题是保证加载到RAM中。

---

Edit:

Read this.

#4

One option would be to create a RAM Disk out of your host's memory. While there is no longer native support for this in the distributed Windows code, you can still find the drivers necessary for free or made available through commercial products. For instance, DRDataRam provides a free driver for personal use and a commercially licensed product for business use at: http://memory.dataram.com/products-and-services/software/ramdisk

一种选择是从主机的内存中创建一个RAM磁盘。虽然在分布式Windows代码中不再提供本机支持，但您仍然可以找到免费或通过商业产品提供的驱动程序。例如，DRDataRam提供一个供个人使用的免费驱动程序，以及一个商业许可产品，用于商业用途:http://memory.dataram.com/products- services/software/ramdisk

There is also ImDisk Virtual Driver available at: http://www.ltr-data.se/opencode.html/#ImDisk It is open sourced and free for commercial use. It is digitally signed with a trusted certificate from Microsoft.

也有ImDisk虚拟驱动程序可以在:http://www.ltr-data.se/opencode.html/#ImDisk获得，它是开源的，商业使用是免费的。它是由微软的可信证书进行数字签名的。

For more information concerning RAM Drives on Windows, check out ServerFault.com.

有关Windows上的RAM驱动器的更多信息，请参阅ServerFault.com。

#5

You should take a look at Address Windowing Extensions (AWE). It sounds like it matches the memory constraints you have (emphasis mine):

你应该看看地址窗口扩展(敬畏)。这听起来好像和你的内存限制相匹配(强调我的):

AWE uses physical nonpaged memory and window views of various portions of this physical memory within a 32-bit virtual address space.

AWE在32位虚拟地址空间中使用物理非分页内存和此物理内存的不同部分的窗口视图。