[20170927]hugepages与内核参数nr_overcommit_hugepages.txt

[20170927]hugepages与内核参数nr_overcommit_hugepages.txt

/proc/sys/vm/nr_overcommit_hugepages specifies how large the pool of huge pages can grow, if more huge pages than
/proc/sys/vm/nr_hugepages are requested by applications. Writing any non-zero value into this file indicates that the
hugetlb subsystem is allowed to try to obtain that number of "surplus" huge pages from the kernel's normal page pool,
when the persistent huge page pool is exhausted. As these surplus huge pages become unused, they are freed back to the
kernel's normal page pool.

--//从上面的说明可以看出设置这个参数可以减少hagepages的使用,避免设置过大导致的浪费.
--//我以前提到参数use_large_pages可以设置auto.参考链接:
--//  [20170313]11G use_large_pages参数.txt => http://blog.itpub.net/267265/viewspace-2135210/
--//  [20170516]11G use_large_pages参数2.txt => http://blog.itpub.net/267265/viewspace-2145480/

--//我提到设置参数use_large_pages=auto,虽然/etc/sysctl.conf配置vm.nr_hugepages = 105.
--//但是可以通过oradism命令启动修改内核参数.但是如果关闭数据库,我的测试配置的hugepage不会回收.

--//测试设置nr_overcommit_hugepages会发生什么情况?

1.环境:
# cat /proc/version
Linux version 2.6.39-300.26.1.el5uek ([e-mail]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-52)) #1 SMP Thu Jan 3 18:31:38 PST 2013

# cat /proc/sys/vm/nr_overcommit_hugepages
0
--//缺省是0.

# cat /proc/meminfo | grep -i hugepage
AnonHugePages:         0 kB
HugePages_Total:     305
HugePages_Free:       96
HugePages_Rsvd:       96
HugePages_Surp:      200
Hugepagesize:       2048 kB

--//数据库配置情况:
SCOTT@book> show sga
Total System Global Area  634732544 bytes
Fixed Size                  2255792 bytes
Variable Size             197133392 bytes
Database Buffers          427819008 bytes
Redo Buffers                7524352 bytes

SCOTT@book> show parameter use_large_pages
NAME            TYPE   VALUE
--------------- ------ -----
use_large_pages string AUTO

SYS@book> alter system set use_large_pages=only scope=spfile;
System altered.

--//如果参数为是Only，从含义上，表示Oracle实例只会使用HugePage作为内存使用。如果系统在AMM模式或者HugePage用尽的时候，数
--//据库就不能启动或者报错。

--//关闭数据库,修改/etc/sysctl.conf,再启动看看:
SYS@book> shutdown immediate ;
Database closed.
Database dismounted.
ORACLE instance shut down.

--//修改/etc/sysctl.conf
vm.nr_hugepages = 105
vm.nr_overcommit_hugepages=1000

# sysctl -p

# cat /proc/meminfo | grep -i hugepage
AnonHugePages:         0 kB
HugePages_Total:     105
HugePages_Free:      105
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
--//现在HugePages_Total: 105

--//注意检查修改/etc/security/limits.conf内容:
oracle soft    memlock        unlimited
oracle hard    memlock        unlimited

--//再启动数据库看看:
SYS@book> startup
ORACLE instance started.
Total System Global Area  634732544 bytes
Fixed Size                  2255792 bytes
Variable Size             197133392 bytes
Database Buffers          427819008 bytes
Redo Buffers                7524352 bytes
Database mounted.
Database opened.

--//可以发现现在可以正常启动!!检查alert文件:

Starting ORACLE instance (normal)
Specified value of sga_max_size is too small, bumping to 637534208
************************ Large Pages Information *******************
Parameter use_large_pages = ONLY
Per process system memlock (soft) limit = 51 GB
Total Shared Global Region in Large Pages = 610 MB (100%)
Large Pages used by this instance: 305 (610 MB)
Large Pages unused system wide = 0 (0 KB)
Large Pages configured system wide = 305 (610 MB)
Large Page size = 2048 KB
********************************************************************

--//可以发现我现在设置vm.nr_hugepages = 105,实际上使用305,但是数据库可以正常启动,而且use_large_pages = ONLY,也就是强制使
--//用hugepages.通过如下命令也可以发现:

# cat /proc/meminfo | grep -i hugepage
AnonHugePages:         0 kB
HugePages_Total:     305
HugePages_Free:       97
HugePages_Rsvd:       97
HugePages_Surp:      200
Hugepagesize:       2048 kB

SYS@book> shutdown immediate ;
Database closed.
Database dismounted.
ORACLE instance shut down.

# cat /proc/meminfo | grep -i hugepage
AnonHugePages:         0 kB
HugePages_Total:     105
HugePages_Free:      105
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

--//可以发现使用的hupepages回收了.也就是使用的内存空间回收了.这样的使用方式,可以一定程度避免内存浪费.

3.继续测试现在设置vm.nr_overcommit_hugepages=0.
--//修改/etc/sysctl.conf
vm.nr_hugepages = 105
vm.nr_overcommit_hugepages=0

# sysctl -p
# cat /proc/meminfo | grep -i hugepage
AnonHugePages:         0 kB
HugePages_Total:     105
HugePages_Free:      105
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

SYS@book> startup
ORA-27137: unable to allocate large pages to create a shared memory segment
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 624951296
Additional information: 1

--//可以这样无法获得足够的hugepages,数据库无法启动.修改回来后在启动正常.
vm.nr_overcommit_hugepages=1000

总结:
通过设置这个参数,一定程度避免内存浪费,必须设置大的hugepages,而导致一些内存浪费,而且不使用时可以回收.