- 前言
Floating IP 是相对于Fixed IP而言的，它一般是在VM创建后分配给VM的，可以达到的目的就是，外界可以访问通过这个Floating Ip访问这个VM，VM也可以通过这个IP访问外界。
在OpenStack中，这个Floating IP使用了namespace内的iptables建立NAT 转发机制来达到VM与外界的通讯的。这片文章主要讲述如何使用OpenStack搭建和使用Floating IP.
- Environment Setup
- Ubuntu 14.04 LTS
- 2个网卡，分别是eth0（192.168.1.46） 和 eth1（192.168.2.46，这个interface在switch使用vlanid=100）
- 如果你的机器是VM，请参照http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004099，去开启eth0 和eth1的promisic模式， 这一步很关键，如果没有启用，外界是无法ping通分配给VM的Floating IP的
- Devstack Configuration
- 启用Nova, Neutron, Cinder, Keystone, Glance
- 安装OpenvSwitch，然后做如下配置
# Add eth1 into br-eth1 and set eth0 and eth1 to promisc mode
sudo ovs-vsctl br-exists br-eth1 || sudo ovs-vsctl add-br br-eth1
sudo ovs-vsctl --may-exist add-port br-eth1 eth1
sudo ip link set dev eth1 promisc on
sudo ip link set dev eth0 promisc on
sudo ip addr flush eth1
sudo ip link set dev eth1 up
- Devstack local.conf
# Note: please change XX to your devstack node IP, change VLAN_START and VLAN_END to your reserved vlan range Here
# 我的IP是192.168.1.46， 所在网络是192.168.1.0/24
[[local|localrc]]
# Set API endpoint host using HOST_IP
HOST_IP=XX
# Use to specify the endpoint
SERVICE_HOST=XX ADMIN_PASSWORD=welcome
MYSQL_PASSWORD=welcome
RABBIT_PASSWORD=welcome
SERVICE_PASSWORD=welcome
SERVICE_TOKEN=welcome
disable_service h-eng
disable_service h-api
disable_service h-api-cfn
disable_service h-api-cw
disable_service tempest
#enable_service tempest
disable_service dstat
# Enable Neturon
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron # stack.sh will freshen each repo on each run if RECLONE
# is set to yes
RECLONE=True # Setting OFFLINE=True to enable stack.sh to run multiple
# times without an Internet connection
OFFLINE=False # Set FLAT_INTERFACE to the Ethernet interface that connects
# the host to your local network
FLAT_INTERFACE=eth1
#FLOATING_RANGE=192.168.1.0/
#Q_FLOATING_ALLOCATION_POOL=start=192.168.1.100,end=192.168.1.119
#PUBLIC_NETWORK_GATEWAY=192.168.1.30
PUBLIC_INTERFACE=eth0
PHYSICAL_NETWORK=public_eth1
OVS_PHYSICAL_BRIDGE=br-ex
#PUBLIC_BRIDGE=br-ex FIXED_RANGE=192.168.100./
#FIXED_NETWORK_SIZE=
NETWORK_GATEWAY=192.168.100. # IMAGE_URLS accepts a comma separated list of images to pre-load into OpenStack
IMAGE_URLS=http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
[[post-config|$NOVA_CONF]]
[DEFAULT]
rpc_response_timeout=
service_down_time=
libvirt_iscsi_use_multipath = True
#force_config_drive = False [libvirt]
iscsi_use_multipath = True [database]
max_pool_size=
max_overflow= [[post-config|/$Q_PLUGIN_CONF_FILE]]
[ml2]
tenant_network_types = vlan,flat
[ml2_type_flat]
flat_networks = public_eth0
[ml2_type_vlan]
network_vlan_ranges = public_eth1:VLAN_START:VLAN_END [ovs]
bridge_mappings = public_eth0:br-ex,public_eth1:br-eth1
enable_tunneling = False- Setup flat and vlan network
- 接下来，我要做两件事，一个是创建一个VLAN ID为100的private network，以后的虚拟机默认是从这个网络分配IP的。
- 然后是一个flat的public network，这个网络上是用来分配floating IP
创建private network，VLAN ID为100
stack@openstack-wangp11-:~/devstack$ neutron net-create --provider:network_type vlan --provider:physical_network public_eth1 --provider:segmentation_id vlan_100
stack@openstack-wangp11-:~/devstack$ neutron net-show vlan_100
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | 2426ff0d-953f-467f-a564-c4f63d926836 |
| mtu | |
| name | vlan_100 |
| port_security_enabled | True |
| provider:network_type | vlan |
| provider:physical_network | public_eth1 |
| provider:segmentation_id | |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f |
| tenant_id | 8cb8c084ffb84914b41d5044ecbcad4e |
+---------------------------+--------------------------------------+
stack@openstack-wangp11-:~/devstack$ neutron subnet-create --enable-dhcp --ip-version --name vlan_100_subnet01 2426ff0d-953f-467f-a564-c4f63d926836 192.168.46.0/
stack@openstack-wangp11-:~/devstack$ neutron subnet-show fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
+-------------------+----------------------------------------------------+
| Field | Value |
+-------------------+----------------------------------------------------+
| allocation_pools | {"start": "192.168.46.2", "end": "192.168.46.254"} |
| cidr | 192.168.46.0/ |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 192.168.46.1 |
| host_routes | |
| id | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f |
| ip_version | |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | vlan_100_subnet01 |
| network_id | 2426ff0d-953f-467f-a564-c4f63d926836 |
| subnetpool_id | |
| tenant_id | 8cb8c084ffb84914b41d5044ecbcad4e |
+-------------------+----------------------------------------------------+
接下来，创建Floating IP所在的public network, 网络类型是flat
注意下面：192.168.1.100-192.168.1.119是网络管理员分配给我的IP pool，192.168.1.30是我们网络的router ip
# Step 1: create external flat network
$ neutron net-create --router:external --provider:network_type flat --provider:physical_network public_eth0 public_net
# Step : create subnet based on your reservation Here
$ neutron subnet-create --name public_eth0_subnet01 --allocation-pool start=192.168.1.100,end=192.168.1.119 --gateway 192.168.1.30 --disable-dhcp --ip-version 22d01ce7-b4c4-4af2-bc3c-9c3991903b4c 192.168.1.0/
# Step : create a external router
$ neutron router-create router1
# Step 4: connect router1 with the private subnet via 'neutron router-interface-add <router-id> <private_subnet_id>'
$ neutron router-interface-add 2da62299-6e41-4b24-a1c7-a3d6cc4db1c8 fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
# Step 5: Set the external public network as the router gateway 'neutron router-gateway-set <router-id> <public_subnet_id>'
$ neutron router-gateway-set 73a38db4-20f9-49ba-b855-472be1e2fd45 e0368da6-197e-4c46-bfd1-4897d61b519a
# Step 6: Create floating IP from public network 'neutron floatingip-create <public network id>'
$ neutron floatingip-create 1452712a-acb3--a5ca-1d838eb2feb9
# Step 7: assicate the VM instance's port in private network to the newly created floating ip port
## get the port id of floating IP (in bold)
$ stack@openstack-wangp11-:~$ neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id | fixed_ip_address | floating_ip_address | port_id |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| a2316e0b-6d72-420f-8a23-f8421160d3d4 | | 192.168.1.102 | 75353e16-5a16-452e-b420-0ada719d625c |
+--------------------------------------+------------------+---------------------+--------------------------------------+
## get the port id of private interface attached to VM instance
$ stack@openstack-wangp11-:~$ neutron port-list
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 16a3e39c-48ef-4d4d-bc8a-96b27da7ff3c | | fa::3e:c1:: | {"subnet_id": "e0368da6-197e-4c46-bfd1-4897d61b519a", "ip_address": "192.168.1.100"} |
| 616d03cd-df6a-465f-a23a-b6bf46e55f7a | | fa::3e::2e:7f | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.4"} |
| 713e4541-7d22--b828-94d714d94ec0 | | fa::3e:af::7b | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.2"} |
| 75353e16-5a16-452e-b420-0ada719d625c | | fa::3e:::db | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.3"}
# 上面红色的port是VM在private network的interface
$ stack@openstack-wangp11-:~$ neutron floatingip-associate a2316e0b-6d72-420f-8a23-f8421160d3d4 75353e16-5a16-452e-b420-0ada719d625c
# Now you can use '192.168.1.102' to login to the VM instance from 192.168..XX network
我在最前面提到，这个Floating IP是通过namespace内的iptables规则实现的，如何查看呢？下面是在neutron node上的一个实例，192.168.1.102是Floating IP，192.168.46.3是private IP
ip netns exec qrouter-<private-network-id> iptables -t nat -S -A quantum-l3-agent-OUTPUT -d 192.168.1.102/ -j DNAT --to-destination 192.168.46.3
-A quantum-l3-agent-PREROUTING -d 192.168.1.102/ -j DNAT --to-destination 192.168.46.3
-A quantum-l3-agent-float-snat -s 192.168.46.3/ -j SNAT --to-source 192.168.1.102
- 启用nameserver
现在，VM跟外界可以通过192.168.1.102进行通信了，在外界看来，VM的Ip就是192.168.1.102了，但是有一个问题，无法访问网站，也无法sudo apt-get update
必须要启用nameserver才可以访问外面的网址
# add following to /etc/resolv.conf of the VM instance to enable name service nameserver <your name sever ip>
nameserver <your name server ip > #and run following to enable
sudo resolvconf -u
- 问题诊断
TODO
- 参考文章
https://www.mirantis.com/blog/configuring-floating-ip-addresses-networking-openstack-public-private-clouds/
https://www.rdoproject.org/Networking_in_too_much_detail
https://blogs.oracle.com/ronen/entry/running_openstack_icehouse_with_zfs