我们首先启动一台机器，启动的时候attach一个volume
创建一个空的cinder volume
root:~# cinder create --display-name emptyvolume11g 11
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| created_at | 2014-07-01T22:22:22.802708 |
| display_description | None |
| display_name | emptyvolume11g |
| encrypted | False |
| id | f6ba87f7-d0b6-4fdb-ac82-346371e78c48 |
| metadata | {} |
| size | 11 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| volume_type | None |
+---------------------+--------------------------------------+
root:~# cinder list
+--------------------------------------+-----------+----------------+------+-------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+----------------+------+-------------+----------+-------------+
| f6ba87f7-d0b6-4fdb-ac82-346371e78c48 | available | emptyvolume11g | 11 | None | false | |
+--------------------------------------+-----------+----------------+------+-------------+----------+-------------+
root:~# glance image-list
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
| 3d0f6926-3c74-49d2-9465-801e51635a23 | Cirros | qcow2 | bare | 14680064 | active |
| 6d9d011c-b007-47df-9044-0729d01d2101 | cirros-0.3.2 | qcow2 | bare | 13167616 | active |
| d96b0e41-8264-41de-8dbb-6b31ce9bfbfc | ubuntu-12.04 | qcow2 | bare | 260637184 | active |
| d23d14bb-6156-43b0-b883-926547cfdec4 | ubuntu-14.04 | qcow2 | bare | 484376576 | active |
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
root:~# nova keypair-list
+-----------+-------------------------------------------------+
| Name | Fingerprint |
+-----------+-------------------------------------------------+
| openstack | 00:3a:f1:5f:8b:97:d7:fa:0f:9e:4e:be:54:57:02:25 |
+-----------+-------------------------------------------------+
nova --debug boot --flavor 2 --image d96b0e41-8264-41de-8dbb-6b31ce9bfbfc --key-name openstack --security-groups default --block-device source=volume,id=f6ba87f7-d0b6-4fdb-ac82-346371e78c48,dest=volume,shutdown=preserve myInstanceWithVolume
如果我们ssh到虚拟机里面可以看到attached volume
root@myinstancewithvolume:/home/ubuntu# fdisk -l
Disk /dev/vda: 21.5 GB, 21474836480 bytes
4 heads, 32 sectors/track, 327680 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c917a
Device Boot Start End Blocks Id System
/dev/vda1 * 2048 41943039 20970496 83 Linux
Disk /dev/vdb: 11.8 GB, 11811160064 bytes
16 heads, 63 sectors/track, 22885 cylinders, total 23068672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/vdb doesn't contain a valid partition table
通过virsh dumpxml，也可以看到
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/nova/instances/59ca11ea-0978-4f7d-8385-480649e63a1d/disk'/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-path/ip-16.158.166.197:3260-iscsi-iqn.2010-10.org.openstack:volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48-lun-1'/>
<target dev='vdb' bus='virtio'/>
<serial>f6ba87f7-d0b6-4fdb-ac82-346371e78c48</serial>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</disk>
在cinder volume节点上可以看到Logic volume
root# lvdisplay
--- Logical volume ---
LV Path /dev/cinder-volumes/volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
LV Name volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
VG Name cinder-volumes
LV UUID PKRR9B-nUCI-Ob1Y-U97W-b1Rh-RhuH-Ow4h5P
LV Write Access read/write
LV Creation host, time escto-bj-hp-z620, 2014-07-02 06:22:23 +0800
LV snapshot status source of
_snapshot-138e677d-721a-435e-b945-fbd6009f3b2a [active]
LV Status available
# open 1
LV Size 11.00 GiB
Current LE 2816
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 252:2
--- Logical volume ---
LV Path /dev/cinder-volumes/_snapshot-138e677d-721a-435e-b945-fbd6009f3b2a
LV Name _snapshot-138e677d-721a-435e-b945-fbd6009f3b2a
VG Name cinder-volumes
LV UUID uTu3GB-J1Nb-1G6Z-xroP-nUmk-394l-KHVYIk
LV Write Access read/write
LV Creation host, time escto-bj-hp-z620, 2014-07-02 19:14:40 +0800
LV snapshot status active destination for volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
LV Status available
# open 0
LV Size 11.00 GiB
Current LE 2816
COW-table size 11.00 GiB
COW-table LE 2816
Allocated to snapshot 0.00%
Snapshot chunk size 4.00 KiB
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 252:3
在cinder volume节点上用tgt查看iscsi target
root# tgtadm --mode target --op show
Target 1: iqn.2010-10.org.openstack:volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
System information:
Driver: iscsi
State: ready
I_T nexus information:
I_T nexus: 1
Initiator: iqn.1993-08.org.debian:01:f7337e3e61e4 alias: escto-bj-hp-z620
Connection: 0
IP Address: 16.158.166.197
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 11811 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rdwr
Backing store path: /dev/cinder-volumes/volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
Backing store flags:
Account information:
ACL information:
ALL
下面我们手工attach一个volume
在cinder volume上创建一个iscsi target
tgtadm --lld iscsi --op new --mode target --tid 2 -T iqn.2010-10.org.openstack:volume-cliu8-test
Target 2: iqn.2010-10.org.openstack:volume-cliu8-test
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00020000
SCSI SN: beaf20
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
Account information:
ACL information:
创建一个Logic volume
lvcreate -L 14G -n lv_cliu8_test cinder-volumes
--- Logical volume ---
LV Path /dev/cinder-volumes/lv_cliu8_test
LV Name lv_cliu8_test
VG Name cinder-volumes
LV UUID Fg1uLV-Rdfc-p33c-1m03-GMrn-QgJ4-mpMelP
LV Write Access read/write
LV Creation host, time escto-bj-hp-z620, 2014-07-02 23:20:27 +0800
LV Status available
# open 0
LV Size 14.00 GiB
Current LE 3584
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 252:6
将Logic Volume加入刚才创建的iscsi target
tgtadm --lld iscsi --op new --mode logicalunit --tid 2 --lun 1 -b /dev/cinder-volumes/lv_cliu8_test
Target 2: iqn.2010-10.org.openstack:volume-cliu8-test
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00020000
SCSI SN: beaf20
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00020001
SCSI SN: beaf21
Size: 15032 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rdwr
Backing store path: /dev/cinder-volumes/lv_cliu8_test
Backing store flags:
Account information:
ACL information:
配置iscsi target监听链接
tgtadm --lld iscsi --op bind --mode target --tid 2 -I ALL
root@escto-bj-hp-z620:/var/lib/cinder/volumes# netstat -tulpn | grep 3260
tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN 22569/tgtd
tcp6 0 0 :::3260 :::* LISTEN 22569/tgtd
在虚拟机运行的节点上discover这个iscsi target
iscsiadm --mode discovery --type sendtargets --portal 16.158.166.197
root# iscsiadm --mode discovery --type sendtargets --portal 16.158.166.197
16.158.166.197:3260,1 iqn.2010-10.org.openstack:volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
16.158.166.197:3260,1 iqn.2010-10.org.openstack:volume-cliu8-test
连接那个iscsi target
root# iscsiadm --mode node --targetname iqn.2010-10.org.openstack:volume-cliu8-test --portal 16.158.166.197:3260 --login
Logging in to [iface: default, target: iqn.2010-10.org.openstack:volume-cliu8-test, portal: 16.158.166.197,3260] (multiple)
Login to [iface: default, target: iqn.2010-10.org.openstack:volume-cliu8-test, portal: 16.158.166.197,3260] successful.
这个时候fdisk可以看到disk已经出现在host上了
Disk /dev/sdf: 15.0 GB, 15032385536 bytes
64 heads, 32 sectors/track, 14336 cylinders, total 29360128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdf doesn't contain a valid partition table
将这个disk attach到虚拟机上
root:~# virsh attach-disk instance-0000000b --source /dev/mapper/cinder--volumes-lv_cliu8_test --target vdc
Disk attached successfully
ssh到虚拟机里面，发现又多了一个disk
root@myinstancewithvolume:/home/ubuntu# fdisk -l
Disk /dev/vda: 21.5 GB, 21474836480 bytes
4 heads, 32 sectors/track, 327680 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c917a
Device Boot Start End Blocks Id System
/dev/vda1 * 2048 41943039 20970496 83 Linux
Disk /dev/vdb: 11.8 GB, 11811160064 bytes
10 heads, 25 sectors/track, 92274 cylinders, total 23068672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x72ab3546
Device Boot Start End Blocks Id System
/dev/vdb1 2048 23068671 11533312 83 Linux
Disk /dev/vdc: 15.0 GB, 15032385536 bytes
16 heads, 63 sectors/track, 29127 cylinders, total 29360128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/vdc doesn't contain a valid partition table