openstack ussuri 版本 all-in-one 离线部署

一 摘要

本文使用kolla-ansible 在centos8.1 上离线安装部署openstack ussuri 版本。

docker\ansible\kolla\kolla-ansible 及依赖包 下载地址
链接：https://pan.baidu.com/s/1jVzi2niOkDADl2yKbhm0FQ
提取码：ao9a

二 环境信息

（一）硬件

2.1.1. cpu型号

 model name      : Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz

2.1.2. 内存

128G

2.1.3.硬盘

4.4T

(二) 软件

2.2.1. 操作系统

CentOS Linux release 8.1.1911 (Core)

2.2.2.ansible 版本

ansible 2.9.10

2.2.3 docker 版本

Docker version 18.03.1-ce, build 9ee9f40

2.2.4 kolla 版本

 kolla -10.1.0

2.2.5 kolla-ansible 版本

 kolla-ansible-10.1.0

三 安装说明

本次安装是在内网实施，内网无法访问互联网，将互联中相关资源下载下来，然后导入内网，实施安装。
目前内网已经搭建了dns 服务器，时间服务器，内网centos8.1 yum 源、内网ansible yum 源、内网docker yum 源。
关于相关依赖包，安装包如何下载 后续专文详述。
安装用户：root

四 安装步骤

（一）安装基础工具

4.1.1 配置内网dns

4.1.1.1修改/etc/resolv.conf

# Generated by NetworkManager
nameserver 10.3.157.201

4.1.1.2 修改 /etc/NetworkManager/NetworkManager.conf

添加dns=none,让其不在生成/etc/resolv.conf

[main]
#plugins=ifcfg-rh,ibft
dns=none

使用ansible-playbook 脚本配置dns

4.1.1.3 重启NetworkManager 服务

[root@localhost NetworkManager]# systemctl restart NetworkManager
[root@localhost NetworkManager]#

4.1.1.4 ansible-playbook 脚本

使用ansible-playbookt 脚本，为后面批量后做准备

[dev@10-3-170-32 ussuri]$ cat modifydns.yml
- hosts: ussuri
  tasks:
   - name: register var
     command: date +%Y-%m-%d_%H-%M-%S
     register: datetime

   - name: display vars
     debug: msg="{{datetime.stdout}}"

   - name: check /etc/resolv.conf
     shell: ls /etc/resolv.conf
     register: resolvmsg
     ignore_errors: true

   - name: file not exist
     when: resolvmsg is failed  #调试文件不存在
     debug: msg="resolv.conf is not found..."


   - name: backup old dns file
     when: resolvmsg is success
     command: cp /etc/resolv.conf  /etc/resolv.conf.bak."{{datetime.stdout}}"

   - name: backup old
     command: cp /etc/NetworkManager/NetworkManager.conf  /etc/NetworkManager/NetworkManager.conf.bak."{{datetime.stdout}}"

   - name: copy intranet resolv.conf to server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/etc/resolv.conf",  dest: "/etc/resolv.conf" }


   - name: copy intranet networkmanager.conf to server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/etc/NetworkManager/NetworkManager.conf",  dest: "/etc/NetworkManager/NetworkManager.conf" }
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"

   - name: restart NetworkManager
     command: systemctl restart NetworkManager
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"


   - name: copy intranet networkmanager.conf to server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/home/dev/ansible/files/centos8/NetworkManager.conf",  dest: "/etc/NetworkManager/NetworkManager.conf" }
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"


   - name: restart NetworkManager
     command: systemctl restart NetworkManager
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"


   - name: test ansible own variable
     debug: msg="OS ansible_distribution {{ ansible_distribution }} major version {{ ansible_distribution_major_version }}"

4.1.2.配置内网yum源

主要配置内网centos8.1  docker 18.03  ansible 源，关于内网源如何搭建 以后在详述。

此处注意几点1.centos8.1 与docker 19 兼容有问题，故安装docker 18.03

4.1.2.1 搭建内网yum 源

 此处略，回头专文详述。

4.1.2.2 更新yum 配置文件

 备份 /etc/yum.repos.d 目录下所有配置文件，然后删除这些文件，将新的配置文件复制到该目录。

4.1.2.3 ansible-palybook 脚本实现 yum 配置文件更新

- hosts: ussuri
  tasks:
   - name: register var
     command: date +%Y-%m-%d_%H-%M-%S
     register: datetime

   - name: display vars
     debug: msg="{{datetime.stdout}}"

   - name: backup old yum file
     command: tar -zcvf  /etc/yum.repos.d."{{datetime.stdout}}".tgz /etc/yum.repos.d

   - name: list the files of  /etc/yum.repos.d
     shell: ls
     args:
      chdir: /etc/yum.repos.d
     register: files_list

   - name: remove files
     file:
       path: /etc/yum.repos.d/{{ item }}
       state: absent
     with_items:
       - "{{ files_list.stdout_lines }}"
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"

   - name: remove files
     file:
       path: /etc/yum.repos.d/{{ item }}
       state: absent
     with_items:
       - "{{ files_list.stdout_lines }}"
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"

   - name: copy intranet yum to 731611 version server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/home/dev/ansible/files/yumconfig/CentOS-73.repo",  dest: "/etc/yum.repos.d/CentOS-73.repo" }
     when: ansible_distribution == "CentOS" and ansible_distribution_version == "7.3"

   - name: copy intranet yum to 771908 version server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/home/dev/ansible/files/yumconfig/CentOS-77.repo",  dest: "/etc/yum.repos.d/CentOS-77.repo" }
     when: ansible_distribution == "CentOS" and ansible_distribution_version == "7.7"

   - name: copy intranet yum to 7 version server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/home/dev/ansible/files/yumconfig/openstack.repo",  dest: "/etc/yum.repos.d/openstack.repo" }
       - { src: "/home/dev/ansible/files/yumconfig/ansible.repo",  dest: "/etc/yum.repos.d/ansible.repo" }
       - { src: "/home/dev/ansible/files/yumconfig/docker.repo",  dest: "/etc/yum.repos.d/docker.repo" }
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "7"

   - name: copy intranet yum to centos8.1 version server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/home/dev/ansible/files/yumconfig/CentOS-81.repo",  dest: "/etc/yum.repos.d/CentOS-81.repo" }
     when: ansible_distribution == "CentOS" and ansible_distribution_version == "8.1"

   - name: copy intranet yum to 8 version server
     copy:
       src: "{{ item.src }}"
       dest: "{{ item.dest }}"
     with_items:
       - { src: "/home/dev/ansible/files/yumconfig/ansible2910forcentos8.repo",  dest: "/etc/yum.repos.d/ansible2910forcentos8.repo" }
       - { src: "/home/dev/ansible/files/yumconfig/docker1803forcentos8.repo",  dest: "/etc/yum.repos.d/docker1803forcentos8.repo" }
     when: ansible_distribution == "CentOS" and ansible_distribution_major_version == "8"

4.1.3 配置域名

配置域名为ussuricontroller1.chouniu.fun
在dns 服务器上配置即可。

4.1.4 修改机器名为 ussuricontroller1.chouniu.fun

hostnamectl set-hostname ussuricontroller1.cloud.chouniu.fun

4.1.5 安装ansible

使用yum 安装ansible 2.9.10，我这里已配置了内网yum 源，为后面批量安装考虑

4.1.5.1 安装命令

[root@ussuricontroller1 ~]# yum install ansible

4.1.5.2 安装结果检查

[root@ussuricontroller1 ~]# ansible --version
ansible 2.9.10
  config file = /etc/ansible/ansible.cfg
  configured module search path = [\'/root/.ansible/plugins/modules\', \'/usr/share/ansible/plugins/modules\']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.8 (default, Nov 21 2019, 19:31:34) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)]
[root@ussuricontroller1 ~]#

4.1.5.3 配置文件优化

对ansible /etc/ansible/ansible.cfg 配置文件做了优化
先备份原文件：cp /etc/ansible/ansible.cfg /etc/ansible/ansible.cfg.bak.orig

新增如下内容：

[defaults]
inventory  = $HOME/ansible/hosts
host_key_checking=False
pipelining=True
forks=10

4.1.5.4 安装过程中问题

遇到这两个错误 nothing provides module(perl:5.26) needed by module perl-DBD-SQLite:1.58:8010020191114033549:073fa5fe-0.x86_64，直接忽略

[root@ussuricontroller1 ~]# yum install ansible
Last metadata expiration check: 0:07:33 ago on Tue 03 Nov 2020 05:47:54 PM CST.
Modular dependency problems:

 Problem 1: conflicting requests
  - nothing provides module(perl:5.26) needed by module perl-DBD-SQLite:1.58:8010020191114033549:073fa5fe-0.x86_64
 Problem 2: conflicting requests
  - nothing provides module(perl:5.26) needed by module perl-DBI:1.641:8010020191113222731:16b3ab4d-0.x86_64
Dependencies resolved.

4.1.6 安装docker

centos8.1 目前适配docker 18.03, 安装docker 19 能装上，但是后面兼容有问题。

4.1.6.1 安装命令

yum install docker-ce
[root@ussuricontroller1 ansible]# yum install docker-ce

4.1.6.2 安装过程中问题

包冲突，删除该包即可。

file /usr/share/man/man1/docker.1.gz from install of docker-ce-18.03.1.ce-1.el7.centos.x86_64 conflicts with file from package podman-manpage s-1.4.2-5.module_el8.1.0+237+63e26edc.noarch

  file /usr/share/man/man1/docker-wait.1.gz from install of docker-ce-18.03.1.ce-1.el7.centos.x86_64 conflicts with file from package podman-ma    npages-1.4.2-5.module_el8.1.0+237+63e26edc.noarch
  file /usr/share/man/man1/docker.1.gz from install of docker-ce-18.03.1.ce-1.el7.centos.x86_64 conflicts with file from package podman-manpage    s-1.4.2-5.module_el8.1.0+237+63e26edc.noarch

[root@ussuricontroller1 ansible]# rpm -qa | grep docker
[root@ussuricontroller1 ansible]# rpm -qa | grep podman
podman-manpages-1.4.2-5.module_el8.1.0+237+63e26edc.noarch
podman-1.4.2-5.module_el8.1.0+237+63e26edc.x86_64
cockpit-podman-4-1.module_el8.1.0+237+63e26edc.noarch
[root@ussuricontroller1 ansible]# yum remove podman

包依赖，可忽略该错误 nothing provides module(perl:5.26) needed by module perl-DBD-SQLite:1.58:8010020191114033549:073fa5fe-0.x86_64

 Problem 1: conflicting requests
  - nothing provides module(perl:5.26) needed by module perl-DBD-SQLite:1.58:8010020191114033549:073fa5fe-0.x86_64
 Problem 2: conflicting requests
  - nothing provides module(perl:5.26) needed by module perl-DBI:1.641:8010020191113222731:16b3ab4d-0.x86_64

4.1.6.3 设置开机启动

[root@ussuricontroller1 ~]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

4.1.6.4 结果验证

[root@ussuricontroller1 ansible]# docker -v
Docker version 18.03.1-ce, build 9ee9f40
[root@ussuricontroller1 ansible]#

4.1.7 安装pip

centos8.1 自带python3，默然安装了pip3.6 ，我们只需要做下链接即可。

4.1.7.1 制作pip链接

[root@ussuricontroller1 ~]# whereis pip3.6
pip3: /usr/bin/pip3 /usr/bin/pip3.6 /usr/share/man/man1/pip3.1.gz
[root@ussuricontroller1 ~]# ln -s /usr/bin/pip3.6 /usr/bin/pip
[root@ussuricontroller1 ~]# pip -V
pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
[root@ussuricontroller1 ~]#

4.1.8 安装基础依赖

centos8 使用dnf 安装rpm 包，安装python3-devel libffi-devel gcc openssl-devel python3-libselinux git vim bash-completion net-tools 基础依赖包

4.1.8.1 安装命令

dnf install python3-devel libffi-devel gcc openssl-devel python3-libselinux git vim bash-completion net-tools

[root@ussuricontroller1 ussuri]# cd dependencies/
[root@ussuricontroller1 dependencies]# ll
total 31224
-rw-r--r--. 1 root root   280084 Jul 28 09:23 bash-completion-2.7-5.el8.noarch.rpm
-rw-r--r--. 1 root root 24564532 Jul 28 09:23 gcc-8.3.1-4.5.el8.x86_64.rpm
-rw-r--r--. 1 root root   190956 Jul 28 09:23 git-2.18.2-2.el8_1.x86_64.rpm
-rw-r--r--. 1 root root    29396 Jul 28 09:23 libffi-devel-3.1-21.el8.i686.rpm
-rw-r--r--. 1 root root    29376 Jul 28 09:23 libffi-devel-3.1-21.el8.x86_64.rpm
-rw-r--r--. 1 root root   330916 Jul 28 09:23 net-tools-2.0-0.51.20160912git.el8.x86_64.rpm
-rw-r--r--. 1 root root  2395376 Jul 28 09:23 openssl-devel-1.1.1c-2.el8_1.1.i686.rpm
-rw-r--r--. 1 root root  2395344 Jul 28 09:23 openssl-devel-1.1.1c-2.el8_1.1.x86_64.rpm
-rw-r--r--. 1 root root    16570 Jul 28 09:23 python36-devel-3.6.8-2.module_el8.1.0+245+c39af44f.x86_64.rpm
-rw-r--r--. 1 root root   290084 Jul 28 09:23 python3-libselinux-2.9-2.1.el8.x86_64.rpm
-rw-r--r--. 1 root root  1427224 Jul 28 09:23 vim-enhanced-8.0.1763-13.el8.x86_64.rpm
[root@ussuricontroller1 dependencies]# dnf install python3-devel libffi-devel gcc openssl-devel python3-libselinux git vim bash-completion net-tools

4.1.9 安装kolla

本次安装的是kolla 10.1.0 版本

4.1.9.1 解压安装包

解压源码包
[root@ussuricontroller1 ussuri]# unzip kolla-10.1.0.zip
解压安装依赖包
tar -zxvf kollapip.tgz

4.1.9.2 安装

git 初始化

[root@ussuricontroller1 ussuri]# cd kolla-10.1.0/
[root@ussuricontroller1 kolla-10.1.0]# git init
Initialized empty Git repository in /root/software/ussuri/kolla-10.1.0/.git/
[root@ussuricontroller1 kolla-10.1.0]#

安装依赖包

[root@ussuricontroller1 kolla-10.1.0]# pip install --no-index --find-links=/root/software/ussuri/kollapip -r /root/software/ussuri/kolla-10.1.0/requirements.txt

安装kolla

[root@ussuricontroller1 kolla-10.1.0]# pip install /root/software/ussuri/kolla-10.1.0/
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Processing /root/software/ussuri/kolla-10.1.0
Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/local/lib/python3.6/site-packages (from kolla==0.0.0)
Requirement already satisfied: docker>=2.4.2 in /usr/local/lib/python3.6/site-packages (from kolla==0.0.0)
Requirement already satisfied: Jinja2>=2.8 in /usr/lib/python3.6/site-packages (from kolla==0.0.0)
Requirement already satisfied: GitPython>=1.0.1 in /usr/local/lib/python3.6/site-packages (from kolla==0.0.0)
Requirement already satisfied: oslo.config>=5.1.0 in /usr/local/lib/python3.6/site-packages (from kolla==0.0.0)
Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/lib/python3.6/site-packages (from docker>=2.4.2->kolla==0.0.0)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/lib/python3.6/site-packages (from docker>=2.4.2->kolla==0.0.0)
Requirement already satisfied: six>=1.4.0 in /usr/lib/python3.6/site-packages (from docker>=2.4.2->kolla==0.0.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib64/python3.6/site-packages (from Jinja2>=2.8->kolla==0.0.0)
Requirement already satisfied: gitdb<5,>=4.0.1 in /usr/local/lib/python3.6/site-packages (from GitPython>=1.0.1->kolla==0.0.0)
Requirement already satisfied: PyYAML>=3.12 in /usr/lib64/python3.6/site-packages (from oslo.config>=5.1.0->kolla==0.0.0)
Requirement already satisfied: oslo.i18n>=3.15.3 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.1.0->kolla==0.0.0)
Requirement already satisfied: debtcollector>=1.2.0 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.1.0->kolla==0.0.0)
Requirement already satisfied: rfc3986>=1.2.0 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.1.0->kolla==0.0.0)
Requirement already satisfied: stevedore>=1.20.0 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.1.0->kolla==0.0.0)
Requirement already satisfied: netaddr>=0.7.18 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.1.0->kolla==0.0.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests!=2.18.0,>=2.14.2->docker>=2.4.2->kolla==0.0.0)
Requirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python3.6/site-packages (from requests!=2.18.0,>=2.14.2->docker>=2.4.2->kolla==0.0.0)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests!=2.18.0,>=2.14.2->docker>=2.4.2->kolla==0.0.0)
Requirement already satisfied: smmap<4,>=3.0.1 in /usr/local/lib/python3.6/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.1->kolla==0.0.0)
Requirement already satisfied: wrapt>=1.7.0 in /usr/local/lib64/python3.6/site-packages (from debtcollector>=1.2.0->oslo.config>=5.1.0->kolla==0.0.0)
Installing collected packages: kolla
  Running setup.py install for kolla ... done
Successfully installed kolla-0.0.0

4.1.9.3 安装验证

能正常输出kolla-build 命令，即表示安装成功。

[root@ussuricontroller1 kolla-10.1.0]# kolla-build -h

4.1.10 安装kolla-ansible

本文安装的是kolla-ansible 10.1.0 版本

4.1.10.1 解压安装包

解压源码包

[root@ussuricontroller1 ussuri]# unzip kolla-ansible-10.1.0.zip
Archive:  kolla-ansible-10.1.0.zip
6bba8cc52af3a26678da48129856f80c21eb8e38

解压依赖包

[root@ussuricontroller1 ussuri]# tar -zxvf kollaansiblepip.tgz

4.1.10.2 安装

git 初始化

[root@ussuricontroller1 ussuri]# cd kolla-ansible-10.1.0/
[root@ussuricontroller1 kolla-ansible-10.1.0]# git init
Initialized empty Git repository in /root/software/ussuri/kolla-ansible-10.1.0/.git/
[root@ussuricontroller1 kolla-ansible-10.1.0]#

安装kolla-ansible 依赖包

[root@ussuricontroller1 kolla-ansible-10.1.0]# pip install --no-index --find-links=/root/software/ussuri//kollaansiblepip -r /root/software/ussuri/kolla-ansible-10.1.0/requirements.txt

安装kolla-ansible

[root@ussuricontroller1 kolla-ansible-10.1.0]# pip install /root/software/ussuri/kolla-ansible-10.1.0
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Processing /root/software/ussuri/kolla-ansible-10.1.0
Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/local/lib/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: oslo.config>=5.2.0 in /usr/local/lib/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: PyYAML>=3.12 in /usr/lib64/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: cryptography>=2.1 in /usr/lib64/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: oslo.utils>=3.33.0 in /usr/local/lib/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: Jinja2>=2.10 in /usr/lib/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: jmespath>=0.9.3 in /usr/local/lib/python3.6/site-packages (from kolla-ansible==0.0.0)
Requirement already satisfied: netaddr>=0.7.18 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: rfc3986>=1.2.0 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: requests>=2.18.0 in /usr/lib/python3.6/site-packages (from oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: stevedore>=1.20.0 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: debtcollector>=1.2.0 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: oslo.i18n>=3.15.3 in /usr/local/lib/python3.6/site-packages (from oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: idna>=2.1 in /usr/lib/python3.6/site-packages (from cryptography>=2.1->kolla-ansible==0.0.0)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/lib/python3.6/site-packages (from cryptography>=2.1->kolla-ansible==0.0.0)
Requirement already satisfied: six>=1.4.1 in /usr/lib/python3.6/site-packages (from cryptography>=2.1->kolla-ansible==0.0.0)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in /usr/lib64/python3.6/site-packages (from cryptography>=2.1->kolla-ansible==0.0.0)
Requirement already satisfied: pytz>=2013.6 in /usr/lib/python3.6/site-packages (from oslo.utils>=3.33.0->kolla-ansible==0.0.0)
Requirement already satisfied: netifaces>=0.10.4 in /usr/lib64/python3.6/site-packages (from oslo.utils>=3.33.0->kolla-ansible==0.0.0)
Requirement already satisfied: iso8601>=0.1.11 in /usr/local/lib/python3.6/site-packages (from oslo.utils>=3.33.0->kolla-ansible==0.0.0)
Requirement already satisfied: pyparsing>=2.1.0 in /usr/local/lib/python3.6/site-packages (from oslo.utils>=3.33.0->kolla-ansible==0.0.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib64/python3.6/site-packages (from Jinja2>=2.10->kolla-ansible==0.0.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests>=2.18.0->oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests>=2.18.0->oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: wrapt>=1.7.0 in /usr/local/lib64/python3.6/site-packages (from debtcollector>=1.2.0->oslo.config>=5.2.0->kolla-ansible==0.0.0)
Requirement already satisfied: pycparser in /usr/lib/python3.6/site-packages (from cffi!=1.11.3,>=1.7->cryptography>=2.1->kolla-ansible==0.0.0)
Installing collected packages: kolla-ansible
  Running setup.py install for kolla-ansible ... done
Successfully installed kolla-ansible-0.0.0
[root@ussuricontroller1 kolla-ansible-10.1.0]#

4.1.10.3 验证

正常使用kolla-ansible 命令 即可

[root@ussuricontroller1 kolla-ansible-10.1.0]# kolla-ansible -h
Usage: /usr/local/bin/kolla-ansible COMMAND [options]

4.1.10.4 新增kollaansible 配置文件

[root@ussuricontroller1 kolla-ansible-10.1.0]# mkdir -p /etc/kolla
[root@ussuricontroller1 kolla-ansible-10.1.0]#  cp -r /root/software/ussuri/kolla-ansible-10.1.0/etc/kolla/* /etc/kolla
[root@ussuricontroller1 kolla-ansible-10.1.0]# ll /etc/kolla/
total 36
-rw-r--r--. 1 root root 25509 Nov  4 10:21 globals.yml
-rw-r--r--. 1 root root  5037 Nov  4 10:21 passwords.yml
[root@ussuricontroller1 kolla-ansible-10.1.0]#

4.1.10.5 新增ansbile 配置文件

[root@ussuricontroller1 kolla-ansible-10.1.0]# mkdir /root/ansible
[root@ussuricontroller1 kolla-ansible-10.1.0]# cp /root/software/ussuri/kolla-ansible-10.1.0/ansible/inventory/* /root/ansible/
[root@ussuricontroller1 kolla-ansible-10.1.0]# ll /root/ansible/
total 24
-rw-r--r--. 1 root root  9584 Nov  4 10:22 all-in-one
-rw-r--r--. 1 root root 10058 Nov  4 10:22 multinode
[root@ussuricontroller1 kolla-ansible-10.1.0]#

4.1.11 关闭防火墙

[root@ussuricontroller1 kolla]# systemctl stop firewalld && systemctl disable firewalld &&systemctl status firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead) since Wed 2020-11-04 12:22:21 CST; 3min 59s ago
     Docs: man:firewalld(1)
 Main PID: 1806 (code=exited, status=0/SUCCESS)

Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -t filter -F DOCKER-ISO>
Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -t filter -X DOCKER-ISO>
Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -t filter -F DOCKER-ISO>
Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -t filter -X DOCKER-ISO>
Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -D FORWARD -i docker0 ->
Nov 04 12:03:00 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -w --table filter --del>
Nov 04 12:03:00 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -w --table filter --del>
Nov 04 12:03:00 ussuricontroller1.cloud.kxdigit.com firewalld[1806]: WARNING: COMMAND_FAILED: \'/usr/sbin/iptables -w10 -w --table filter --del>
Nov 04 12:22:19 ussuricontroller1.cloud.kxdigit.com systemd[1]: Stopping firewalld - dynamic firewall daemon...
Nov 04 12:22:21 ussuricontroller1.cloud.kxdigit.com systemd[1]: Stopped firewalld - dynamic firewall daemon.
lines 1-16/16 (END)

4.1.12 永久关闭selinux

修改文件/etc/selinux/config SELINUX=disabled，并重启服务器

[root@ussuricontroller1 selinux]# getenforce
Enforcing
[root@ussuricontroller1 selinux]# pwd
/etc/selinux
[root@ussuricontroller1 selinux]# cp /etc/selinux/config /etc/selinux/config.bak.orig
[root@ussuricontroller1 selinux]# vi /etc/selinux/config
[root@ussuricontroller1 selinux]#

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

(二) openstack all-in-one 安装

下面主要介绍使用kolla-ansible 安装openstack ussuri 版本

4.2.1 修改docker 配置

4.2.1.1 修改docker 启动文件

新增 /etc/systemd/system/docker.service.d/kolla.conf

[root@ussuricontroller1 ~]# cd /etc/systemd/system/
[root@ussuricontroller1 system]# mkdir docker.service.d
[root@ussuricontroller1 system]# cd docker.service.d/
[root@ussuricontroller1 docker.service.d]# vi kolla.conf
[root@ussuricontroller1 docker.service.d]# ll

文件内容

[Service]
MountFlags=shared

重启docker

[root@ussuricontroller1 docker.service.d]# systemctl daemon-reload
[root@ussuricontroller1 docker.service.d]# systemctl restart docker
[root@ussuricontroller1 docker.service.d]#

4.2.2 生成passwords.yml

使用命令生成openstack 部署过程中 使用的密码，然后手工修改下登录密码

4.2.2.1 密码生成

[root@ussuricontroller1 ~]# kolla-genpwd
[root@ussuricontroller1 ~]# cd /etc/kolla/
[root@ussuricontroller1 kolla]# ll
total 56
-rw-r--r--. 1 root root 25509 Nov  4 10:21 globals.yml
-rw-r--r--. 1 root root 25667 Nov  4 11:24 passwords.yml
[root@ussuricontroller1 kolla]#

4.2.2.2 修改登录密码

keystone_admin_password: Admin_PASS

keepalived_password: YHr5RibiOPjHnzAV6pkHzKLvVpMxnE0mGT8AI6X3
keystone_admin_password: Admin_PASS
keystone_database_password: sEAtChpWzuRZDggAcwhArxeMk26GAoBiX9ZVxeOw

4.2.3 修改/etc/kolla/globals.yml

备份原文件，然后修改相关配置

4.2.3.1 all-in-one 配置修改

kolla_internal_vip_address: "10.3.176.8" 使用 你的服务器所在网络里未使用的IP

docker_registry: registry.chouniu.fun 使用你自己的docker 仓库地址

kolla_base_distro: "centos"
kolla_install_type: "source"
openstack_release: "ussuri"
kolla_internal_vip_address: "10.3.176.8"
network_interface: "eth0"
neutron_external_interface: "eth1"
docker_registry: registry.chouniu.fun

4.2.4 kolla-ansible 预检查

4.2.4.1 预检查

[root@ussuricontroller1 kolla]# kolla-ansible prechecks
Pre-deployment checking : ansible-playbook -i /usr/local/share/kolla-ansible/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla  -e kolla_action=precheck /usr/local/share/kolla-ansible/ansible/site.yml

4.2.4.2 问题排查

根据预检查报错，解决相关问题

报错
TASK [nova-cell : Checking that libvirt is not running]

解决 禁用 libvirt

[root@ussuricontroller1 kolla]# systemctl stop libvirtd.service && systemctl disable libvirtd.service && systemctl status libvirtd.service
Removed /etc/systemd/system/multi-user.target.wants/libvirtd.service.
Removed /etc/systemd/system/sockets.target.wants/virtlogd.socket.
Removed /etc/systemd/system/sockets.target.wants/virtlockd.socket.
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; disabled; vendor preset: enabled)
   Active: inactive (dead) since Wed 2020-11-04 11:41:39 CST; 154ms ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 1923 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/libvirtd.service
           ├─2666 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leases>
           └─2668 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leases>

Nov 02 17:15:51 localhost.localdomain dnsmasq-dhcp[2666]: DHCP, sockets bound exclusively to interface virbr0
Nov 02 17:15:51 localhost.localdomain dnsmasq[2666]: reading /etc/resolv.conf
Nov 02 17:15:51 localhost.localdomain dnsmasq[2666]: using nameserver 10.3.157.201#53
Nov 02 17:15:51 localhost.localdomain dnsmasq[2666]: read /etc/hosts - 2 addresses
Nov 02 17:15:51 localhost.localdomain dnsmasq[2666]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Nov 02 17:15:51 localhost.localdomain dnsmasq-dhcp[2666]: read /var/lib/libvirt/dnsmasq/default.hostsfile
Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com dnsmasq[2666]: reading /etc/resolv.conf
Nov 04 11:02:38 ussuricontroller1.cloud.kxdigit.com dnsmasq[2666]: using nameserver 10.3.157.201#53
Nov 04 11:41:39 ussuricontroller1.cloud.kxdigit.com systemd[1]: Stopping Virtualization daemon...
Nov 04 11:41:39 ussuricontroller1.cloud.kxdigit.com systemd[1]: Stopped Virtualization daemon.
lines 1-20/20 (END)

4.2.4.3 预检查成功标识

成功会在 显示failed=0

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=96   changed=0    unreachable=0    failed=0    skipped=188  rescued=0    ignored=0

[root@ussuricontroller1 kolla]#

4.2.4.4 虚机部署openstack(物理机跳过该步骤)

如果你是在虚机上部署openstack ，请务必修改如下配置,

[root@webmysql1 kolla]# mkdir -p /etc/kolla/config/nova

[root@webmysql1 nova]# vi nova-compute.conf
[root@webmysql1 nova]# cat nova-compute.conf
[libvirt]
virt_type = qemu
cpu_mode = none
[root@webmysql1 nova]#

4.2.5 all-in-one 部署

本次使用all-in-one 部署

部署命令

kolla-ansible deploy

部署成功

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=371  changed=214  unreachable=0    failed=0    skipped=228  rescued=0    ignored=1

前端访问
http://10.3.176.8

登录后页面

五 使用命令

（一）硬件配置相关

5.1.1 看cpu 型号

cat /proc/cpuinfo | grep \'model name\' | uniq

[root@localhost tmp]# cat /proc/cpuinfo | grep \'model name\' | uniq
model name      : Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
[root@localhost tmp]#

5.1.2 看内存

cat /proc/meminfo | grep MemTotal

[root@localhost tmp]# cat /proc/meminfo | grep MemTotal
MemTotal:       131631220 kB

5.1.3看硬盘

fdisk -l

[root@localhost tmp]# fdisk -l
Disk /dev/sda: 4.4 TiB, 4788878049280 bytes, 9353277440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 9E1FBEBA-2C06-4D17-AEFD-D0ACD51C87A6

Device         Start        End    Sectors  Size Type
/dev/sda1       2048       4095       2048    1M BIOS boot
/dev/sda2       4096    2101247    2097152    1G Linux filesystem
/dev/sda3    2101248  421531647  419430400  200G Linux filesystem
/dev/sda4  421531648  455086079   33554432   16G Linux swap
/dev/sda5  455086080 9353275391 8898189312  4.1T Linux filesystem
[root@localhost tmp]#

六 问题

（一）cpu兼容性问题
Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz，不支持ussuri 版 制作实例。
报错点：
error: qemu unexpectedly closed the monitor: 2020-11-06T03:37:10.776902Z qemu-kvm: error: failed to set MSR 0x48e to 0xfff9fffe04006172
qemu-kvm: /builddir/build/BUILD/qemu-4.2.0/target/i386/kvm.c:2695: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs\' failed.

详细报错信息

: libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-11-06T03:37:10.776902Z qemu-kvm: error: failed to set MSR 0x
48e to 0xfff9fffe04006172
2020-11-06 11:37:11.382 7 ERROR nova.virt.libvirt.driver [req-7cce33c6-4d75-4c32-be71-36795d640728 f22dadfe806e4844ac9441c09d00faa1 c7b8d6a3737
140a2addb28a398b2448f - default default] [instance: 7aa22ba5-54c0-4703-beba-f3dc69ddbb15] Failed to start libvirt guest: libvirt.libvirtError:
internal error: qemu unexpectedly closed the monitor: 2020-11-06T03:37:10.776902Z qemu-kvm: error: failed to set MSR 0x48e to 0xfff9fffe0400617
2
2020-11-06 11:37:11.541 7 INFO os_vif [req-7cce33c6-4d75-4c32-be71-36795d640728 f22dadfe806e4844ac9441c09d00faa1 c7b8d6a3737140a2addb28a398b244
8f - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:43:41:62,bridge_name=\'qbr463437af-c8\',has_traffic_filt
ering=True,id=463437af-c8f5-4a18-b59c-3549c9a7d25c,network=Network(4a447859-7077-4035-b817-bc45702a810c),plugin=\'ovs\',port_profile=VIFPortProfi
leOpenVSwitch,preserve_on_delete=False,vif_name=\'tap463437af-c8\')
2020-11-06 11:37:11.541 7 INFO nova.virt.libvirt.driver [req-7cce33c6-4d75-4c32-be71-36795d640728 f22dadfe806e4844ac9441c09d00faa1 c7b8d6a37371
40a2addb28a398b2448f - default default] [instance: 7aa22ba5-54c0-4703-beba-f3dc69ddbb15] Deleting instance files /var/lib/nova/instances/7aa22b
a5-54c0-4703-beba-f3dc69ddbb15_del
2020-11-06 11:37:11.542 7 INFO nova.virt.libvirt.driver [req-7cce33c6-4d75-4c32-be71-36795d640728 f22dadfe806e4844ac9441c09d00faa1 c7b8d6a37371
40a2addb28a398b2448f - default default] [instance: 7aa22ba5-54c0-4703-beba-f3dc69ddbb15] Deletion of /var/lib/nova/instances/7aa22ba5-54c0-4703
-beba-f3dc69ddbb15_del complete
2020-11-06 11:37:11.625 7 ERROR nova.compute.manager [req-7cce33c6-4d75-4c32-be71-36795d640728 f22dadfe806e4844ac9441c09d00faa1 c7b8d6a3737140a
2addb28a398b2448f - default default] [instance: 7aa22ba5-54c0-4703-beba-f3dc69ddbb15] Instance failed to spawn: libvirt.libvirtError: internal
error: qemu unexpectedly closed the monitor: 2020-11-06T03:37:10.776902Z qemu-kvm: error: failed to set MSR 0x48e to 0xfff9fffe04006172
qemu-kvm: /builddir/build/BUILD/qemu-4.2.0/target/i386/kvm.c:2695: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs\' failed.