Floating IP in OpenStack Neutron

时间:2022-05-31 07:03:44
  • 前言

Floating IP 是相对于Fixed IP而言的,它一般是在VM创建后分配给VM的,可以达到的目的就是,外界可以访问通过这个Floating Ip访问这个VM,VM也可以通过这个IP访问外界。

在OpenStack中,这个Floating IP使用了namespace内的iptables建立NAT 转发机制来达到VM与外界的通讯的。这片文章主要讲述如何使用OpenStack搭建和使用Floating IP.

  • Environment Setup
  1. Ubuntu 14.04 LTS
  2. 2个网卡,分别是eth0(192.168.1.46) 和 eth1(192.168.2.46,这个interface在switch使用vlanid=100)
  3. 如果你的机器是VM,请参照http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004099,去开启eth0 和eth1的promisic模式, 这一步很关键,如果没有启用,外界是无法ping通分配给VM的Floating IP的
  • Devstack Configuration
  1. 启用Nova, Neutron, Cinder, Keystone, Glance
  2. 安装OpenvSwitch,然后做如下配置
  3. # Add eth1 into br-eth1 and set eth0 and eth1 to promisc mode
    sudo ovs-vsctl br-exists br-eth1 || sudo ovs-vsctl add-br br-eth1
    sudo ovs-vsctl --may-exist add-port br-eth1 eth1
    sudo ip link set dev eth1 promisc on
    sudo ip link set dev eth0 promisc on
    sudo ip addr flush eth1
    sudo ip link set dev eth1 up
  • Devstack local.conf
  • #  Note: please change XX to your devstack node IP,  change VLAN_START and VLAN_END to your reserved vlan range Here
    # 我的IP是192.168.1.46, 所在网络是192.168.1.0/24
    [[local|localrc]]
    # Set API endpoint host using HOST_IP
    HOST_IP=XX
    # Use to specify the endpoint
    SERVICE_HOST=XX ADMIN_PASSWORD=welcome
    MYSQL_PASSWORD=welcome
    RABBIT_PASSWORD=welcome
    SERVICE_PASSWORD=welcome
    SERVICE_TOKEN=welcome
    disable_service h-eng
    disable_service h-api
    disable_service h-api-cfn
    disable_service h-api-cw
    disable_service tempest
    #enable_service tempest
    disable_service dstat
    # Enable Neturon
    disable_service n-net
    enable_service q-svc
    enable_service q-agt
    enable_service q-dhcp
    enable_service q-l3
    enable_service q-meta
    enable_service neutron # stack.sh will freshen each repo on each run if RECLONE
    # is set to yes
    RECLONE=True # Setting OFFLINE=True to enable stack.sh to run multiple
    # times without an Internet connection
    OFFLINE=False # Set FLAT_INTERFACE to the Ethernet interface that connects
    # the host to your local network
    FLAT_INTERFACE=eth1
    #FLOATING_RANGE=192.168.1.0/
    #Q_FLOATING_ALLOCATION_POOL=start=192.168.1.100,end=192.168.1.119
    #PUBLIC_NETWORK_GATEWAY=192.168.1.30
    PUBLIC_INTERFACE=eth0
    PHYSICAL_NETWORK=public_eth1
    OVS_PHYSICAL_BRIDGE=br-ex
    #PUBLIC_BRIDGE=br-ex FIXED_RANGE=192.168.100./
    #FIXED_NETWORK_SIZE=
    NETWORK_GATEWAY=192.168.100. # IMAGE_URLS accepts a comma separated list of images to pre-load into OpenStack
    IMAGE_URLS=http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
    [[post-config|$NOVA_CONF]]
    [DEFAULT]
    rpc_response_timeout=
    service_down_time=
    libvirt_iscsi_use_multipath = True
    #force_config_drive = False [libvirt]
    iscsi_use_multipath = True [database]
    max_pool_size=
    max_overflow= [[post-config|/$Q_PLUGIN_CONF_FILE]]
    [ml2]
    tenant_network_types = vlan,flat
    [ml2_type_flat]
    flat_networks = public_eth0
    [ml2_type_vlan]
    network_vlan_ranges = public_eth1:VLAN_START:VLAN_END [ovs]
    bridge_mappings = public_eth0:br-ex,public_eth1:br-eth1
    enable_tunneling = False
  • Setup flat and vlan network​
  1. 接下来,我要做两件事,一个是创建一个VLAN ID为100的private network,以后的虚拟机默认是从这个网络分配IP的。
  2. 然后是一个flat的public network,这个网络上是用来分配floating IP

创建private network,VLAN ID为100

stack@openstack-wangp11-:~/devstack$ neutron net-create --provider:network_type vlan  --provider:physical_network public_eth1 --provider:segmentation_id  vlan_100​
stack@openstack-wangp11-:~/devstack$ neutron net-show vlan_100
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | 2426ff0d-953f-467f-a564-c4f63d926836 |
| mtu | |
| name | vlan_100 |
| port_security_enabled | True |
| provider:network_type | vlan |
| provider:physical_network | public_eth1 |
| provider:segmentation_id | |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f |
| tenant_id | 8cb8c084ffb84914b41d5044ecbcad4e |
+---------------------------+--------------------------------------+
stack@openstack-wangp11-:~/devstack$ neutron subnet-create --enable-dhcp --ip-version --name vlan_100_subnet01 2426ff0d-953f-467f-a564-c4f63d926836 192.168.46.0/​
stack@openstack-wangp11-:~/devstack$ neutron subnet-show fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
+-------------------+----------------------------------------------------+
| Field | Value |
+-------------------+----------------------------------------------------+
| allocation_pools | {"start": "192.168.46.2", "end": "192.168.46.254"} |
| cidr | 192.168.46.0/ |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 192.168.46.1 |
| host_routes | |
| id | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f |
| ip_version | |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | vlan_100_subnet01 |
| network_id | 2426ff0d-953f-467f-a564-c4f63d926836 |
| subnetpool_id | |
| tenant_id | 8cb8c084ffb84914b41d5044ecbcad4e |
+-------------------+----------------------------------------------------+

接下来,创建Floating IP所在的public network, 网络类型是flat

注意下面:192.168.1.100-192.168.1.119是网络管理员分配给我的IP pool,192.168.1.30是我们网络的router ip

# Step 1: create external flat network
$ neutron net-create --router:external --provider:network_type flat --provider:physical_network public_eth0 public_net
# Step : create subnet based on your reservation Here
$ neutron subnet-create --name public_eth0_subnet01 --allocation-pool start=192.168.1.100,end=192.168.1.119 --gateway​ 192.168.1.30 --disable-dhcp --ip-version 22d01ce7-b4c4-4af2-bc3c-9c3991903b4c 192.168.1.0/
# Step : create a external router
$ neutron router-create router1
# Step 4: connect router1 with the private subnet via 'neutron router-interface-add <router-id> <private_subnet_id>'
$ neutron router-interface-add 2da62299-6e41-4b24-a1c7-a3d6cc4db1c8 fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
# Step 5: Set the external public network as the router gateway 'neutron router-gateway-set <router-id> <public_subnet_id>'
$ neutron router-gateway-set 73a38db4-20f9-49ba-b855-472be1e2fd45 e0368da6-197e-4c46-bfd1-4897d61b519a
# Step 6: Create floating IP from public network 'neutron floatingip-create <public network id>'
$ neutron floatingip-create 1452712a-acb3--a5ca-1d838eb2feb9
# Step 7: assicate the VM instance's port in private network to the newly created floating ip port
## get the port id of floating IP (in bold)
$ stack@openstack-wangp11-:~$ neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id | fixed_ip_address | floating_ip_address | port_id |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| a2316e0b-6d72-420f-8a23-f8421160d3d4 | | 192.168.1.102 | 75353e16-5a16-452e-b420-0ada719d625c |
+--------------------------------------+------------------+---------------------+--------------------------------------+
## get the port id of private interface attached to VM instance
$ stack@openstack-wangp11-:~$ neutron port-list
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 16a3e39c-48ef-4d4d-bc8a-96b27da7ff3c | | fa::3e:c1:: | {"subnet_id": "e0368da6-197e-4c46-bfd1-4897d61b519a", "ip_address": "192.168.1.100"} |
| 616d03cd-df6a-465f-a23a-b6bf46e55f7a | | fa::3e::2e:7f | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.4"} |
| 713e4541-7d22--b828-94d714d94ec0 | | fa::3e:af::7b | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.2"} |
| 75353e16-5a16-452e-b420-0ada719d625c | | fa::3e:::db | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.3"}

# 上面红色的port是VM在private network的interface
$ stack@openstack-wangp11-:~$ neutron floatingip-associate a2316e0b-6d72-420f-8a23-f8421160d3d4 75353e16-5a16-452e-b420-0ada719d625c
# Now you can use '192.168.1.102' to login to the VM instance from 192.168..XX network

我在最前面提到,这个Floating IP是通过namespace内的iptables规则实现的,如何查看呢?下面是在neutron node上的一个实例,192.168.1.102是Floating IP,192.168.46.3是private IP

ip netns exec qrouter-<private-network-id> iptables -t nat -S

-A quantum-l3-agent-OUTPUT -d 192.168.1.102/ -j DNAT --to-destination 192.168.46.3
-A quantum-l3-agent-PREROUTING -d 192.168.1.102/ -j DNAT --to-destination 192.168.46.3
-A quantum-l3-agent-float-snat -s 192.168.46.3/ -j SNAT --to-source 192.168.1.102
  • 启用nameserver

现在,VM跟外界可以通过192.168.1.102进行通信了,在外界看来,VM的Ip就是192.168.1.102了,但是有一个问题,无法访问网站,也无法sudo apt-get update

必须要启用nameserver才可以访问外面的网址

# add following to /etc/resolv.conf of the VM instance to enable name service

nameserver <your name sever ip>
nameserver <your name server ip > #and run following to enable
sudo resolvconf -u
  • 问题诊断

TODO

  • 参考文章

https://www.mirantis.com/blog/configuring-floating-ip-addresses-networking-openstack-public-private-clouds/

https://www.rdoproject.org/Networking_in_too_much_detail

https://blogs.oracle.com/ronen/entry/running_openstack_icehouse_with_zfs