DRBD簡介
Distributed Replicated Block Device(DRBD)是一個用軟體實作的、無共享的、伺服器之間鏡像塊裝置内容的存儲複制解決方案。
資料鏡像:實時、透明、同步(所有伺服器都成功後傳回)、異步(本地伺服器成功後傳回)
DBRD的核心功能通過Linux的核心實作,最接近系統的IO棧,但它不能神奇地添加上層的功能比如檢測到EXT3檔案系統的崩潰。
DBRD的位置處于檔案系統以下,比檔案系統更加靠近作業系統核心及IO棧。
以上資訊,來自于網際網路,請尊重版權。接下來筆者将向您展示如何配置heartbeat+drbd的執行個體。
環境要求:某企業為了實作高可用性的群集檔案系統,出于成本的考慮采用drbd技術。
實驗拓撲:
![](https://img.laitimes.com/img/9ZDMuAjOiMmIsIjOiQnIsIyZuBnL2ATN0QDOwEDMvwVNwITMwIzLcRnbl1GajFGd0F2Lc12bj5yb0NWM14SMn1Wavw1LcpDc0RHaiojIsJye.png)
實驗步驟:
一、準備環境
1.系統環境
# uname –r //核心版本
2.6.18-164.el5
# cat /etc/redhat-release //檢視系統資訊
Red Hat Enterprise Linux Server release 5.4 (Tikanga)
2.修改主機名稱等
在節點1上進行修改
# hostname node1.a.com //修改主機名稱
# vim /etc/sysconfig/network //修改檔案
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=node1.a.com
之後退出再次登陸。
# hostname //檢視名稱
node1.a.com
# vim /etc/hosts //修改hosts檔案
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.1.3 node1.a.com
192.168.1.4 node2.a.com
在節點2上進行修改
# hostname node2.a.com //修改主機名稱
# vim /etc/sysconfig/network //修改檔案
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=node2.a.com
之後退出再次登陸。
# hostname //檢視名稱
node2.a.com
# vim /etc/hosts //修改hosts檔案
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.1.3 node1.a.com
192.168.1.4 node2.a.com
3.安裝所需的檔案
預先将所需的檔案上傳到管理者家目錄。以下操作在node1.a.com和node2.a.com上都進行操作。
為了友善你的使用,筆者已将全部所需檔案上傳到筆者空間,提供您免費下載下傳:http://down.51cto.com/data/401397
# ll
total 3084
-rw------- 1 root root 1291 Feb 8 02:05 anaconda-ks.cfg
-rw-r--r-- 1 root root 221868 May 7 2012 drbd83-8.3.8-1.el5.centos.i386.rpm
-rw-r--r-- 1 root root 1637238 Mar 14 2010 heartbeat-2.1.4-9.el5.i386.rpm
-rw-r--r-- 1 root root 293349 Mar 14 2010 heartbeat-devel-2.1.4-9.el5.i386.rpm
-rw-r--r-- 1 root root 230890 Mar 14 2010 heartbeat-gui-2.1.4-9.el5.i386.rpm
-rw-r--r-- 1 root root 111742 Mar 14 2010 heartbeat-ldirectord-2.1.4-9.el5.i386.rpm
-rw-r--r-- 1 root root 92070 Mar 14 2010 heartbeat-pils-2.1.4-10.el5.i386.rpm
-rw-r--r-- 1 root root 179199 Mar 14 2010 heartbeat-stonith-2.1.4-10.el5.i386.rpm
-rw-r--r-- 1 root root 35236 Feb 8 02:04 install.log
-rw-r--r-- 1 root root 3995 Feb 8 02:02 install.log.syslog
-rw-r--r-- 1 root root 125974 May 7 2012 kmod-drbd83-8.3.8-1.el5.centos.i686.rpm
-rw-r--r-- 1 root root 56817 Mar 14 2010 libnet-1.1.4-3.el5.i386.rpm
-rw-r--r-- 1 root root 92071 Mar 14 2010 perl-MailTools-1.77-1.el5.noarch.rpm
# yum localinstall *.rpm --nogpgcheck –y //使用yum進行安裝可以有效的解決包的依賴性問題。
4.同步時鐘
要求兩個節點上的始終必須要一緻。
# hwclock -s //同步始終
# date //檢視日期
Wed Feb 8 03:38:44 CST 2012
#
5.加載子產品
在兩個節點上都去進行如下操作。
# modprobe drbd //加載drbd子產品
#
# lsmod |grep drbd //檢視子產品
drbd 228528 0
#
6.建立新分區
以下操作要求在node1.a.com和node2.a.com上都進行。同時要求兩個節點所建立的新分區大小一緻。
# fdisk -l //檢視有關情況
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1288 10241437+ 83 Linux
/dev/sda3 1289 1415 1020127+ 82 Linux swap / Solaris
# fdisk /dev/sda
The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n //建立新分區
Command action
e extended
p primary partition (1-4)
p //建立主分區
Selected partition 4
First cylinder (1416-2610, default 1416):
Using default value 1416
Last cylinder or +size or +sizeM or +sizeK (1416-2610, default 2610): +1G //兩個節點的分區大小要一緻。
Command (m for help): p //顯示
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1288 10241437+ 83 Linux
/dev/sda3 1289 1415 1020127+ 82 Linux swap / Solaris
/dev/sda4 1416 1538 987997+ 83 Linux
Command (m for help): w 儲存
# fdisk -l //再次檢視
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1288 10241437+ 83 Linux
/dev/sda3 1289 1415 1020127+ 82 Linux swap / Solaris
/dev/sda4 1416 1538 987997+ 83 Linux
#
# partprobe /dev/sda //重新加載
二、DRBD配置
以下操作要求在節點node1和node2上都做。
1.修改conf配置檔案
# cd /usr/share/doc/drbd83-8.3.8/
#ls
ChangeLog COPYING drbd.conf file.list README
#cp drbd.conf /etc/ 複制配置檔案到相關目錄
cp: overwrite `/etc/drbd.conf'? y
#cd /etc/drbd.d/
# ls
global_common.conf
# cp -p global_common.conf global_common.conf.bak //最好對檔案進行備份
# vim global_common.conf
//第一列是行的序号
1 global {
2 usage-count no;
3 }
4
5 common {
6 protocol C;
7 startup {
8 wfc-timeout 120;
9 degr-wfc-timeout 120;
10 }
11 disk {
12 on-io-error detach;
13 fencing resource-only;
14 }
15 net {
16 cram-hmac-alg "sha1";
17 shared-secret "mydrbdlab";
18 }
19 syncer {
20 rate 100M;
21 }
22 }
2.修改資源檔案
# cd /etc/drbd.d/
# vim web.res
1 resource web {
2 on node1.a.com {
3 device /dev/drbd0;
4 disk /dev/sda4;
5 address 192.168.1.3:7789;
6 meta-disk internal;
7 }
8 on node2.a.com {
9 device /dev/drbd0;
10 disk /dev/sda4;
11 address 192.168.1.4:7789;
12 meta-disk internal;
13 }
14 }
3.建立資源
#drbdadm create-md web
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
4.啟動服務、測試
# service drbd start
Starting DRBD resources: drbdsetup 0 show:5: delay-probe-volume 0k => 0k out of range[4..1048576]k.
# drbdadm adjust web //測試
drbdsetup 0 show:5: delay-probe-volume 0k => 0k out of range [4..1048576]k.
檢視服務的情況
# service drbd status
drbd driver loaded OK; device status:
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by [email protected], 2010-06-04 08:04:16
m:res cs ro ds p mounted fstype
0:web Connected Secondary/Secondary Inconsistent/Inconsistent C
當然也可以使用下面的指令進行檢視。
# drbd-overview
0:web Connected Secondary/Secondary Inconsistent/Inconsistent C r----
5.指定主裝置
以下操作要求隻是在一個節點上進行完成。筆者隻在node1.a.com上進行完成,指定node1為主要的節點
# drbdadm -- --overwrite-data-of-peer primary web //指定node1為資源主節點
# watch -n 1 'cat /proc/drbd' //使用此指令,可以動态的檢視具體的同步過程
# mkfs -t ext3 -L drbdweb /dev/drbd0 //格式化
# mkdir /web
# mount /dev/drbd0 /web //挂載
#
# service drbd status //在node1.a.com上檢視狀态
drbd driver loaded OK; device status:
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by [email protected], 2010-06-04 08:04:16
m:res cs ro ds p mounted fstype
0:web Connected Primary/Secondary UpToDate/UpToDate C /web ext3
#
切換到node2.a.com上進行檢視
# service drbd status
drbd driver loaded OK; device status:
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by [email protected], 2010-06-04 08:04:16
m:res cs ro ds p mounted fstype
0:web Connected Secondary/Primary UpToDate/UpToDate C
#
三、NFS配置
兩台伺服器的nfs配置必須要一緻。
1.修改相關檔案
# vim /etc/exports
/web *(rw,sync,insecure,no_root_squash,no_wdelay)
2.執行相關操作。
#service portmap start && chkconfig portmap on
#service nfs start && chkconfig nfs on
3.修改nfs啟動腳本
# vim /etc/init.d/nfs
122 killproc nfsd –9
四、Heartbeat配置
Heartbeat的配置要求兩個節點的配置一緻。在node1和node2上進行如下操作
1.複制模版檔案
# cd /usr/share/doc/heartbeat-2.1.4/
# cp authkeys ha.cf haresources /etc/ha.d/
2.修改相關的配置資訊
# cd /etc/ha.d/ //切換到相關目錄
# vim ha.cf
24 debugfile /var/log/ha-debug
29 logfile /var/log/ha-log
34 logfacility local0
48 keepalive 2
56 deadtime 10
76 udpport 694
121 bcast eth0 //此行可以添加在任意一行
157 auto_failback off
211 node node1.a.com
212 node node2.a.com
3.修改資源檔案
# echo "node1.a.com IPaddr::192.168.1.1/24/eth0 drbddisk::web Filesystem::/dev/drbd0::/web::ext3 killnfsd">>/etc/ha.d/haresources
4.修改key檔案
# vim authkeys
23 auth 1
24 1 crc
5.手工建立檔案
# cd /etc/ha.d/resource.d/
# echo "killall -9 nfsd ; /etc/init.d/nfs restart ; exit 0" >>/etc/ha.d/resource.d/killnfsd
6.修改配置檔案的權限
# chmod 600 /etc/ha.d/authkeys
# chmod 755 /etc/ha.d/resource.d/killnfsd
7.啟動服務
# service heartbeat start
五、測試
使用用戶端進行測試
1.挂載
#mkdir /mnt/nfs
#mount 192.168.1.1:/web /mnt/nfs
#mount 192.168.1.1:/web /mnt/nfs
2.建立測試腳本檔案
#vim /mnt/test.sh
//此配置腳本檔案的主要作用是為了反複的讀寫操作
while true
do
echo --\>trying touch x:`date`
touch x
echo \<-----done touch x:`date`
echo
sleep 2
done
3.執行測試檔案
#cd /mnt/nfs/
# bash /mnt/test.sh
--->trying touch x:Sun May 6 07:49:20 CST 2012
<-----done touch x:Sun May 6 07:49:20 CST 2012
--->trying touch x:Sun May 6 07:49:22 CST 2012
<-----done touch x:Sun May 6 07:49:22 CST 2012
--->trying touch x:Sun May 6 07:49:24 CST 2012
<-----done touch x:Sun May 6 07:49:24 CST 2012
--->trying touch x:Sun May 6 07:49:26 CST 2012
<-----done touch x:Sun May 6 07:49:26 CST 2012
--->trying touch x:Sun May 6 07:49:28 CST 2012
<-----done touch x:Sun May 6 07:49:30 CST 2012
--->trying touch x:Sun May 6 07:49:32 CST 2012
<-----done touch x:Sun May 6 07:49:32 CST 2012
--->trying touch x:Sun May 6 07:49:34 CST 2012
<-----done touch x:Sun May 6 07:49:34 CST 2012
--->trying touch x:Sun May 6 07:49:36 CST 2012
<-----done touch x:Sun May 6 07:49:36 CST 2012
//讓用戶端一直執行腳本檔案,到伺服器節點node1上關閉heartbeat服務,之後在用戶端會發現丢棄現象,之後檔案系統又恢複正常。
--->trying touch x:Sun May 6 07:50:22 CST 2012
<-----done touch x:Sun May 6 07:50:22 CST 2012
--->trying touch x:Sun May 6 07:50:24 CST 2012
touch: cannot touch `x': Stale NFS file handle
<-----done touch x:Sun May 6 07:50:30 CST 2012
--->trying touch x:Sun May 6 07:50:33 CST 2012
touch: cannot touch `x': Stale NFS file handle
<-----done touch x:Sun May 6 07:50:33 CST 2012
--->trying touch x:Sun May 6 07:50:35 CST 2012
touch: cannot touch `x': Stale NFS file handle
<-----done touch x:Sun May 6 07:50:35 CST 2012
--->trying touch x:Sun May 6 07:50:37 CST 2012
<-----done touch x:Sun May 6 07:50:37 CST 2012
--->trying touch x:Sun May 6 07:50:39 CST 2012
<-----done touch x:Sun May 6 07:50:39 CST 2012