I am using msync in my application on Linux 2.6 to ensure consistency in the event of a crash. I need to thoroughly test my usage of msync but the implementation seems to be flushing all the relevant pages for me. Is there a way to prevent automatic flushing of mmap'd pages onto the disk to expose erroneous usage of msync on my part?
我在Linux 2.6上的应用程序中使用msync来确保崩溃时的一致性。我需要彻底测试我对msync的使用,但实现似乎正在为我刷新所有相关页面。有没有办法阻止mmap的页面自动刷新到磁盘上,以暴露我的错误使用msync?
2 个解决方案
#1
7
With apologies to @samold, "swappiness" has nothing to do with this. Swappiness just affects how the kernel trades off swapping dirty anonymous pages versus evicting page cache pages when memory is low.
向@samold道歉,“swappiness”与此无关。 Swappiness只影响内核交换掉脏脏的匿名页面与内存不足时驱逐页面缓存页面的方式。
You need to play with the Linux VM tunables controlling the pdflush task. For starters, I would suggest:
您需要使用控制pdflush任务的Linux VM可调参数。对于初学者,我建议:
sysctl -w vm.dirty_writeback_centisecs=360000
By default, vm.dirty_writeback_centisecs is 3000, which means the kernel will consider any dirty page older than 30 seconds to be "too old" and try to flush it to disk. By cranking it up to 1 hour, you should be able to avoid flushing dirty pages to disk at all, at least during a short test. Except...
默认情况下,vm.dirty_writeback_centisecs为3000,这意味着内核会将超过30秒的任何脏页视为“太旧”并尝试将其刷新到磁盘。通过将其启动最多1小时,您应该能够避免将脏页刷到磁盘上,至少在短暂的测试中是这样。除...
sysctl -w vm.dirty_background_ratio=80
By default, vm.dirty_background_ratio is 10, as in 10 percent. That means when more than 10 percent of physical memory is occupied by dirty pages, the kernel will think it needs to get busy flushing something to disk, even if it is younger than dirty_writeback_centisecs. Crank this one up to 80 or 90 and the kernel should be willing to tolerate most of RAM being occupied by dirty pages. (I would not set this too high, though, since I bet nobody ever does that and it might trigger strange behavior.) Except...
默认情况下,vm.dirty_background_ratio为10,如10%。这意味着当脏页面占用超过10%的物理内存时,内核会认为它需要忙于将某些内容刷新到磁盘,即使它比dirty_writeback_centisecs更年轻。将这个加到80或90,内核应该愿意容忍脏页占用的大部分RAM。 (不过,我不会把它设得太高,因为我敢打赌,没有人会这样做,这可能会引发奇怪的行为。)除了......
sysctl -w vm.dirty_ratio=90
By default, vm.dirty_ratio is 40, which means once 40% of RAM is dirty pages, processes attempting to create more dirty pages will block until something gets evicted. Always make this one bigger than dirty_background_ratio. Hm, come to think of it, set this one before that one, just to make sure this one is always larger.
默认情况下,vm.dirty_ratio为40,这意味着一旦40%的RAM是脏页,尝试创建更多脏页的进程将阻塞,直到某些内容被驱逐。总是让这个比dirty_background_ratio更大。嗯,来想一想,在那个之前设置这个,只是为了确保这个总是更大。
That's it for my initial suggestions. It is possible that your kernel will start evicting pages anyway; the Linux VM is a mysterious beast and seems to get tweaked on every release. Hopefully this provides a starting point.
这是我最初的建议。无论如何,你的内核可能会开始驱逐页面; Linux VM是一个神秘的野兽,似乎在每个版本都进行了调整。希望这提供了一个起点。
See Documentation/sysctl/vm.txt in the kernel sources for a complete list of VM tunables. (Preferably refer to the documentation for the kernel version you are actually using.)
有关VM可调参数的完整列表,请参阅内核源中的Documentation / sysctl / vm.txt。 (最好参考您实际使用的内核版本的文档。)
Finally, use the /proc/PID/pagemap interface to see which pages are actually dirty at any time.
最后,使用/ proc / PID / pagemap界面随时查看哪些页面实际上是脏的。
#2
0
A few guesses:
一些猜测:
You can fiddle with the swappiness of the system via the /proc/sys/vm/swappiness tunable:
您可以通过/ proc / sys / vm / swappiness可调参数调整系统的swappiness:
/proc/sys/vm/swappiness
The value in this file controls how aggressively the
kernel will swap memory pages. Higher values increase
agressiveness, lower values descrease aggressiveness.
The default value is 60.
(Wow. proc(5) needs to be run through a spell-checker.)
(哇.proc(5)需要通过拼写检查器运行。)
If setting the swappiness to 0 doesn't do the trick, there are more tunable knobs; the Documentation/laptops/laptop-mode.txt file contains a good description of the laptop_mode script's behaviors:
如果将swappiness设置为0并不能解决问题,那么可以使用更多的可调旋钮; Documentation / laptops / laptop-mode.txt文件包含laptop_mode脚本行为的良好描述:
To increase the effectiveness of the laptop_mode strategy, the laptop_mode
control script increases dirty_expire_centisecs and dirty_writeback_centisecs in
/proc/sys/vm to about 10 minutes (by default), which means that pages that are
dirtied are not forced to be written to disk as often. The control script also
changes the dirty background ratio, so that background writeback of dirty pages
is not done anymore. Combined with a higher commit value (also 10 minutes) for
ext3 or ReiserFS filesystems (also done automatically by the control script),
this results in concentration of disk activity in a small time interval which
occurs only once every 10 minutes, or whenever the disk is forced to spin up by
a cache miss. The disk can then be spun down in the periods of inactivity.
You might wish to take these numbers to their extremes; if you're really curious about your application's behavior, it sounds reasonable to set these values quite high and see how long a sync(1) command takes when it's all done. But these are system-wide tunables -- other applications may not be so happy.
您可能希望将这些数字置于极端;如果您对应用程序的行为非常好奇,将这些值设置得相当高并且看到sync(1)命令完成时需要多长时间是合理的。但这些是系统范围内的可调参数 - 其他应用程序可能不那么开心。
#1
7
With apologies to @samold, "swappiness" has nothing to do with this. Swappiness just affects how the kernel trades off swapping dirty anonymous pages versus evicting page cache pages when memory is low.
向@samold道歉,“swappiness”与此无关。 Swappiness只影响内核交换掉脏脏的匿名页面与内存不足时驱逐页面缓存页面的方式。
You need to play with the Linux VM tunables controlling the pdflush task. For starters, I would suggest:
您需要使用控制pdflush任务的Linux VM可调参数。对于初学者,我建议:
sysctl -w vm.dirty_writeback_centisecs=360000
By default, vm.dirty_writeback_centisecs is 3000, which means the kernel will consider any dirty page older than 30 seconds to be "too old" and try to flush it to disk. By cranking it up to 1 hour, you should be able to avoid flushing dirty pages to disk at all, at least during a short test. Except...
默认情况下,vm.dirty_writeback_centisecs为3000,这意味着内核会将超过30秒的任何脏页视为“太旧”并尝试将其刷新到磁盘。通过将其启动最多1小时,您应该能够避免将脏页刷到磁盘上,至少在短暂的测试中是这样。除...
sysctl -w vm.dirty_background_ratio=80
By default, vm.dirty_background_ratio is 10, as in 10 percent. That means when more than 10 percent of physical memory is occupied by dirty pages, the kernel will think it needs to get busy flushing something to disk, even if it is younger than dirty_writeback_centisecs. Crank this one up to 80 or 90 and the kernel should be willing to tolerate most of RAM being occupied by dirty pages. (I would not set this too high, though, since I bet nobody ever does that and it might trigger strange behavior.) Except...
默认情况下,vm.dirty_background_ratio为10,如10%。这意味着当脏页面占用超过10%的物理内存时,内核会认为它需要忙于将某些内容刷新到磁盘,即使它比dirty_writeback_centisecs更年轻。将这个加到80或90,内核应该愿意容忍脏页占用的大部分RAM。 (不过,我不会把它设得太高,因为我敢打赌,没有人会这样做,这可能会引发奇怪的行为。)除了......
sysctl -w vm.dirty_ratio=90
By default, vm.dirty_ratio is 40, which means once 40% of RAM is dirty pages, processes attempting to create more dirty pages will block until something gets evicted. Always make this one bigger than dirty_background_ratio. Hm, come to think of it, set this one before that one, just to make sure this one is always larger.
默认情况下,vm.dirty_ratio为40,这意味着一旦40%的RAM是脏页,尝试创建更多脏页的进程将阻塞,直到某些内容被驱逐。总是让这个比dirty_background_ratio更大。嗯,来想一想,在那个之前设置这个,只是为了确保这个总是更大。
That's it for my initial suggestions. It is possible that your kernel will start evicting pages anyway; the Linux VM is a mysterious beast and seems to get tweaked on every release. Hopefully this provides a starting point.
这是我最初的建议。无论如何,你的内核可能会开始驱逐页面; Linux VM是一个神秘的野兽,似乎在每个版本都进行了调整。希望这提供了一个起点。
See Documentation/sysctl/vm.txt in the kernel sources for a complete list of VM tunables. (Preferably refer to the documentation for the kernel version you are actually using.)
有关VM可调参数的完整列表,请参阅内核源中的Documentation / sysctl / vm.txt。 (最好参考您实际使用的内核版本的文档。)
Finally, use the /proc/PID/pagemap interface to see which pages are actually dirty at any time.
最后,使用/ proc / PID / pagemap界面随时查看哪些页面实际上是脏的。
#2
0
A few guesses:
一些猜测:
You can fiddle with the swappiness of the system via the /proc/sys/vm/swappiness tunable:
您可以通过/ proc / sys / vm / swappiness可调参数调整系统的swappiness:
/proc/sys/vm/swappiness
The value in this file controls how aggressively the
kernel will swap memory pages. Higher values increase
agressiveness, lower values descrease aggressiveness.
The default value is 60.
(Wow. proc(5) needs to be run through a spell-checker.)
(哇.proc(5)需要通过拼写检查器运行。)
If setting the swappiness to 0 doesn't do the trick, there are more tunable knobs; the Documentation/laptops/laptop-mode.txt file contains a good description of the laptop_mode script's behaviors:
如果将swappiness设置为0并不能解决问题,那么可以使用更多的可调旋钮; Documentation / laptops / laptop-mode.txt文件包含laptop_mode脚本行为的良好描述:
To increase the effectiveness of the laptop_mode strategy, the laptop_mode
control script increases dirty_expire_centisecs and dirty_writeback_centisecs in
/proc/sys/vm to about 10 minutes (by default), which means that pages that are
dirtied are not forced to be written to disk as often. The control script also
changes the dirty background ratio, so that background writeback of dirty pages
is not done anymore. Combined with a higher commit value (also 10 minutes) for
ext3 or ReiserFS filesystems (also done automatically by the control script),
this results in concentration of disk activity in a small time interval which
occurs only once every 10 minutes, or whenever the disk is forced to spin up by
a cache miss. The disk can then be spun down in the periods of inactivity.
You might wish to take these numbers to their extremes; if you're really curious about your application's behavior, it sounds reasonable to set these values quite high and see how long a sync(1) command takes when it's all done. But these are system-wide tunables -- other applications may not be so happy.
您可能希望将这些数字置于极端;如果您对应用程序的行为非常好奇,将这些值设置得相当高并且看到sync(1)命令完成时需要多长时间是合理的。但这些是系统范围内的可调参数 - 其他应用程序可能不那么开心。