kvm guest 设置hugepages

时间：2023-01-04 15:56:03

Host支持两种大小的hugepage：2 MiB, 1 GiB. 默认使用的页面大小是4 KiB. Hugepage会立即分配，并不会swap out。

2MiB 的hugepage

  1. 设置host的大页，使用virsh命令即可，然后guest就可以启动了：
#　virsh allocpages 2M 1024

如果不指定node，此命令会将这1024个hugepage平均分配到host的每个node； 因为默认HugeTLB filesystem已经mount到host，所以不用再做mount。

# mount | grep -i huge
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)

启动虚拟机。hugepage size不指定的话，默认使用2M大小的大页：

# virsh dumpxml test --xpath //memoryBacking
<memoryBacking>
  <hugepages/>
  <access mode="shared"/>
</memoryBacking>  ## 没有numa node的设置
# virsh start test 
Domain 'test' started

qemu 命令行：

-object '{"qom-type":"memory-backend-file","id":"pc.ram","mem-path":"/dev/hugepages/libvirt/qemu/1-test","share":true,"x-use-canonical-path-for-ramblock-id":false,"prealloc":true,"size":2147483648}' \

1GiB 的hugepage

  1. 设置host的大页，并尝试启动虚拟机（失败）：
# virsh allocpages 1G 3
## 查看每个node的1G大页的情况，也可以用virsh capabilities来看：
# virsh freepages  0 1G
1048576KiB: 2
# virsh freepages  1 1G
1048576KiB: 1
# virsh dumpxml rhel --xpath //memoryBacking
<memoryBacking>
  <hugepages>
    <page size="1048576" unit="KiB"/>
  </hugepages>
</memoryBacking>
# virsh start rhel
error: Failed to start domain 'rhel'
error: internal error: Unable to find any usable hugetlbfs mount for 1048576 KiB
## 说明系统默认mount的HugeTLB filesystem只是针对2M的大页，对1G的大页，需要手动设置
  1. 设置1G hugepage的mount point，并restart libvirtd，guest就可以启动了：
# mkdir /dev/hugepages1G
# mount -t hugetlbfs -o pagesize=1G hugetlbfs /dev/hugepages1G
# mount | grep huge
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)
hugetlbfs on /dev/hugepages1G type hugetlbfs (rw,relatime,seclabel,pagesize=1024M)
# virsh start rhel
error: Failed to start domain 'rhel'
error: internal error: Unable to find any usable hugetlbfs mount for 1048576 KiB
# systemctl restart virtqemud
# virsh start rhel
Domain 'rhel' started

### qemu 命令行：
-m 2048 \
-overcommit mem-lock=off \
-smp 8,sockets=8,cores=1,threads=1 \
-object '{"qom-type":"memory-backend-file","id":"ram-node0","mem-path":"/dev/hugepages1G/libvirt/qemu/1-rhel","prealloc":true,"size":2147483648}' \
-numa node,nodeid=0,cpus=0-7,memdev=ram-node0 \
###
标签：

相关文章