配置LIO iSCSI Target
現在主流Linux都可以設定iSCSI,如STGT/TGT、LIO Target等。Linux-IO(LIO)Target是目前Linux标準的iSCSI target的開源實作,包含在Linux核心(2.6.38後)中。LIO不發起會話,而是提供LUNs,等待來自SCSI initiator的指令并進行所需的I/O資料傳輸。更詳細的參考:http://en.wikipedia.org/wiki/LIO_Target
設定一個target,包含以下元件:
儲存設備:使用Linux作為target,可以使用LVM邏輯卷來作為底層儲存設備,也可以使用其他儲存設備,如分區、硬碟、稀疏檔案等。
LUN ID:每個通過iSCSI target共享的儲存設備是當做一個LUN來共享,每個LUN都有一個唯一ID。
Target ID:如果想要授權target連接配接到特定節點,建立不同的target,每個target有自己的target ID,也稱為IQN(Internet Qualifiied Name)。iSCSI用戶端使用IQN來進行連接配接。
Identifier:辨別符用來識别特定的iSCSI target。
端口号:target監聽的TCP端口,預設是3260。
CentOS7上使用targetcli指令來設定iSCSI target.
# systemctl enable target.service
# systemctl start target.service
targetcli支援不同的後端存儲,當建立iSCSI磁盤時,必須指定後端存儲類型。targetcli來檢視可用的後端存儲。
# targetcli
targetcli shell version 2.1.fb34
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> ls
o- / ............................................................................... [...]
o- backstores .................................................................... [...]
| o- block ........................................................ [Storage Objects: 0]
| o- fileio ....................................................... [Storage Objects: 0]
| o- pscsi ........................................................ [Storage Objects: 0]
| o- ramdisk ...................................................... [Storage Objects: 0]
o- iscsi .................................................................. [Targets: 0]
o- loopback ............................................................... [Targets: 0]
添加LVM,使用指令“/backstores/fileio create lun1 /opt/disk1.img 100M”可以添加一個稀疏檔案的target。
/> /backstores/block create lun0 /dev/rootvg/image
Created block storage object lun0 using /dev/rootvg/image.
/> ls
o- / ............................................................................... [...]
o- backstores .................................................................... [...]
| o- block ........................................................ [Storage Objects: 1]
| | o- lun0 ........................ [/dev/rootvg/image (5.0GiB) write-thru deactivated]
| o- fileio ....................................................... [Storage Objects: 0]
| o- pscsi ........................................................ [Storage Objects: 0]
| o- ramdisk ...................................................... [Storage Objects: 0]
o- iscsi .................................................................. [Targets: 0]
o- loopback ............................................................... [Targets: 0]
定義target
/> /iscsi create
Created target iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625.
Created TPG 1.
檢視建立的TPG
/> ls
o- / ............................................................................... [...]
o- backstores .................................................................... [...]
| o- block ........................................................ [Storage Objects: 1]
| | o- lun0 ........................ [/dev/rootvg/image (5.0GiB) write-thru deactivated]
| o- fileio ....................................................... [Storage Objects: 0]
| o- pscsi ........................................................ [Storage Objects: 0]
| o- ramdisk ...................................................... [Storage Objects: 0]
o- iscsi .................................................................. [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625 ............ [TPGs: 1]
| o- tpg1 ..................................................... [no-gen-acls, no-auth]
| o- acls ................................................................ [ACLs: 0]
| o- luns ................................................................ [LUNs: 0]
| o- portals .......................................................... [Portals: 0]
o- loopback ............................................................... [Targets: 0]
使用預設配置建立portal
/> cd /iscsi/iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625/tpg1/
/iscsi/iqn.20...60959625/tpg1> portals/ create
Using default IP port 3260
Binding to INADDR_ANY (0.0.0.0)
Created network portal 0.0.0.0:3260.
/iscsi/iqn.20...60959625/tpg1>
指定LUN到portal,如果想要對LUN進行通路限制,使用iSCSI initiator的IQN,然後檢視目前設定,最終saveconfig儲存。
Created network portal 0.0.0.0:3260.
/iscsi/iqn.20...60959625/tpg1> luns/ create /backstores/block/lun0
Created LUN 0.
/iscsi/iqn.20...60959625/tpg1> acls/ create iqn.2014-03.com.example:123456789
Created Node ACL for iqn.2014-03.com.example:123456789
Created mapped LUN 0.
/iscsi/iqn.20...60959625/tpg1> ls
o- tpg1 ........................................................... [no-gen-acls, no-auth]
o- acls ...................................................................... [ACLs: 1]
| o- iqn.2014-03.com.example:123456789 ................................ [Mapped LUNs: 1]
| o- mapped_lun0 .............................................. [lun0 block/lun0 (rw)]
o- luns ...................................................................... [LUNs: 1]
| o- lun0 ............................................. [block/lun0 (/dev/rootvg/image)]
o- portals ................................................................ [Portals: 1]
o- 0.0.0.0:3260 ................................................................. [OK]
/iscsi/iqn.20...60959625/tpg1> cd /
/> ls
o- / ............................................................................... [...]
o- backstores .................................................................... [...]
| o- block ........................................................ [Storage Objects: 1]
| | o- lun0 .......................... [/dev/rootvg/image (5.0GiB) write-thru activated]
| o- fileio ....................................................... [Storage Objects: 0]
| o- pscsi ........................................................ [Storage Objects: 0]
| o- ramdisk ...................................................... [Storage Objects: 0]
o- iscsi .................................................................. [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625 ............ [TPGs: 1]
| o- tpg1 ..................................................... [no-gen-acls, no-auth]
| o- acls ................................................................ [ACLs: 1]
| | o- iqn.2014-03.com.example:123456789 .......................... [Mapped LUNs: 1]
| | o- mapped_lun0 ........................................ [lun0 block/lun0 (rw)]
| o- luns ................................................................ [LUNs: 1]
| | o- lun0 ....................................... [block/lun0 (/dev/rootvg/image)]
| o- portals .......................................................... [Portals: 1]
| o- 0.0.0.0:3260 ........................................................... [OK]
o- loopback ............................................................... [Targets: 0]
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
連接配接到iSCSI SAN
使用iscsiadm指令來連接配接到iSCSI target。iscsiadm有不同模式,每一個模式來處理iSCSI連接配接的不同階段。
• discoverydb/discovery: 用來查詢iSCSI target并找出可用target。
• node:登入到特定的iSCSI target。
• session: 從目前會話擷取資訊或者建立一個新的會話到已經連接配接的target。
• iface和host: 允許指定連接配接到特定target的方式。
當連接配接到iSCSI target後,會自動建立配置檔案,機器重新開機後,伺服器會自動連接配接到以前的iSCSI連接配接。在用iscsiadm連接配接iSCSI target前,需要確定相關module已經加載。
第一步:Discovery模式
使用iscsiadm --mode discovery --type sendtargets --portal xxxx:3260 --discover來發現iSCSI target。
# iscsiadm --mode discovery --type sendtargets --portal 10.0.0.11:3260 --discover
10.0.0.11:3260,1 iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625
上面的指令不僅顯示target的名字,并且将iSCSI配置添加到/var/lib/iscsi/send_targets目錄下(redhat)。可以使用“-P”(0或1代表debug級别)選項來輸出儲存的資訊。從下面可以看出是使用SENDTARGETS發現類型。
# iscsiadm --mode discoverydb -P 1
SENDTARGETS:
DiscoveryAddress: 10.0.0.11,3260
Target: iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625
Portal: 10.0.0.11:3260,1
Iface Name: default
iSNS:
No targets found.
STATIC:
No targets found.
FIRMWARE:
No targets found.
根據不同的SAN環境,所有的發現類型都可用,iSNS可以設定一個iSNS伺服器,它集中注冊了iSCSI target。fireware是使用iSCSI硬體擴充卡可以發現iSCSI target。SLP目前不可用。
第二步:node模式
在執行上一步後,可以知道taget的IQN。下一步登入target來建立連接配接。node模式是iSCSI術語,表示iSCSI target和特定的portal之間的實際連接配接。portal是連接配接到iSCSI target的位址和端口。
# iscsiadm --mode node --targetname iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625 --portal 10.0.0.11:3260 --login
Logging in to [iface: default, target: iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625, portal: 10.0.0.11,3260] (multiple)
Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625, portal: 10.0.0.11,3260] successful.
使用lsscsi指令可以驗證連接配接,發現新增了LIO-ORG類型的/dev/sdb。
# lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda
<span style="color:#ff6600;">[3:0:0:0] disk LIO-ORG lun0 4.0 /dev/sdb </span>
在建立連接配接後,iSCSI initiator會将配置寫到$ISCSI_ROOT/nodes目錄。default檔案包含了連接配接到iSCSI target的設定。
# ls -lR
.:
total 0
drw-------. 3 root root 29 Nov 7 03:41 iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625
./iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625:
total 0
drw-------. 2 root root 20 Nov 7 03:41 10.0.0.11,3260,1
./iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625/10.0.0.11,3260,1:
total 4
-rw-------. 1 root root 2064 Nov 7 03:41 default
第三步:管理iSCSI連接配接
檢視目前連接配接資訊:
# iscsiadm --mode node -P 1
Target: iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625
Portal: 10.0.0.11:3260,1
Iface Name: default
擷取更詳細的資訊:
# iscsiadm --mode session -P 2
Target: iqn.2003-01.org.linux-iscsi.controller.x8664:sn.bb2460959625 (non-flash)
Current Portal: 10.0.0.11:3260,1
Persistent Portal: 10.0.0.11:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:c4e61affa28
Iface IPaddress: 10.0.0.31
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 1
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
*********
Timeouts:
*********
Recovery Timeout: 120
Target Reset Timeout: 30
LUN Reset Timeout: 30
Abort Timeout: 15
*****
CHAP:
*****
username: <empty>
password: ********
username_in: <empty>
password_in: ********
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 262144
FirstBurstLength: 65536
MaxBurstLength: 262144
ImmediateData: Yes
InitialR2T: Yes
MaxOutstandingR2T: 1
使用iscsi存儲
在成功建立連接配接後,通過fdisk -l可以發現新磁盤,可以當普通的本地磁盤使用。
# fdisk -l
Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000b68ef
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 41943039 20458496 8e Linux LVM
Disk /dev/mapper/centos_compute-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos_compute-root: 18.8 GB, 18798870528 bytes, 36716544 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
<span style="color:#ff6600;">Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes</span>
斷開連接配接會話
iSCSI預設設定在重新開機後伺服器後自動重連。如果需要斷開連接配接,必須移走配置。通過iscsiadm --mode node --logout來斷開會話,這個指令會斷開所有連接配接,同時删除對應配置。