Redhat 5.4 RAC multipath 配置raw，運作root.sh 時報錯Failed to upgrade Oracle Cluster Registry configuration

原文：http://blog.csdn.net/tianlesoftware/article/details/6039210

之前打算裝Oracle 11gR2的RAC的。 後來因為資源不夠，就改裝oracle 10g了。 raw 是用openfiler 配置 加 Multipath多路徑配置的。

       raw的配置參考我的Blog：

              Linux 5.4 LVM RAW 裝置配置的深入研究

這種情況下配置有些注意事項。 否則會在執行root.sh指令的時候，報錯：

[root@rac1 init.d]# /u01/app/oracle/product/crs/root.sh

WARNING: directory '/u01/app/oracle/product' is not owned by root

WARNING: directory '/u01/app/oracle' is not owned by root

WARNING: directory '/u01/app' is not owned by root

WARNING: directory '/u01' is not owned by root

Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory

Setting up NS directories

PROT-1: Failed to initialize ocrconfig

Failed to upgrade Oracle Cluster Registry configuration

一般出現這個問題的原因有幾種：

（1） Bug

       這種情況參考：

       Executing root.sh errors with "Failed To Upgrade Oracle Cluster Registry Configuration" [ID 466673.1]

（2） raw 裝置的權限問題

              [root@rac1 raw]# pwd

              /dev/raw

              [root@rac1 raw]# ll

              total 0

              crw-r----- 1 root   oinstall 162, 1 Nov 25 15:34 raw1

              crw-r----- 1 root   oinstall 162, 2 Nov 25 15:34 raw2

              crw-r--r-- 1 oracle oinstall 162, 3 Nov 25 13:48 raw3

              crw-r--r-- 1 oracle oinstall 162, 4 Nov 25 13:48 raw4

              crw-r--r-- 1 oracle oinstall 162, 5 Nov 25 13:48 raw5

權限要設定成如下：

chown root.oinstall /dev/raw/raw[1-2]

chown oracle.oinstall /dev/raw/raw[3-5]

chmod 775 /dev/raw/raw[1-2] 

chmod 775 /dev/raw/raw[3-5]

可以在udev的規則裡設定， 也可以把權限添加到/etc/rc.d/rc.local  中。

（3） raw 裝置讀寫問題

       Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1 [ID 293819.1]             

清空一下RAW:

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp6 bs=1M count=200

dd: writing `/dev/mapper/crsp6': No space left on device

197+0 records in

196+0 records out

205599744 bytes (206 MB) copied, 21.1817 seconds, 9.7 MB/s

說明：

       這裡有個提示資訊，磁盤空間不足: No space left on device。這裡的情況是，我的每個raw是200M。但是實際容量并沒有這麼多。是以我清空寫200M的時候，會報空間不足。這種情況會按照實際大小進行清空。從下面我們可以該raw裝置實際大小是197M。

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp6 bs=1M count=190

190+0 records in

190+0 records out

199229440 bytes (199 MB) copied, 21.9521 seconds, 9.1 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp5 bs=1M count=195

195+0 records in

195+0 records out

204472320 bytes (204 MB) copied, 22.0362 seconds, 9.3 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp7 bs=1M count=195

204472320 bytes (204 MB) copied, 19.9692 seconds, 10.2 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp8 bs=1M count=195

204472320 bytes (204 MB) copied, 18.5526 seconds, 11.0 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp9 bs=1M count=195

204472320 bytes (204 MB) copied, 21.9439 seconds, 9.3 MB/s

我這次遇到問題的Log如下：

[root@rac1 client]# pwd

/u01/app/oracle/product/crs/log/rac1/client

[root@rac1 client]# ls

ocrconfig_7527.log

[root@rac1 client]# cat ocrconfig_7527.log

Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle.  All rights reserved.

2010-11-25 19:54:14.694: [ OCRCONF][1189568]ocrconfig starts...

2010-11-25 19:54:14.696: [ OCRCONF][1189568]Upgrading OCR data

2010-11-25 19:54:15.434: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:15.464: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:15.464: [  OCRRAW][1189568]proprioini: both disks are not OCR formatted

2010-11-25 19:54:15.464: [  OCRRAW][1189568]proprinit: Could not open raw device

2010-11-25 19:54:15.465: [ default][1189568]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 19:54:15.465: [ OCRCONF][1189568]Exporting OCR data to [OCRUPGRADEFILE]

2010-11-25 19:54:15.465: [  OCRAPI][1189568]a_init:7!: Backend init unsuccessful : [33]

2010-11-25 19:54:15.465: [ OCRCONF][1189568]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]

2010-11-25 19:54:16.072: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.075: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.075: [  OCRRAW][1189568]proprioini: both disks are not OCR formatted

2010-11-25 19:54:16.075: [  OCRRAW][1189568]proprinit: Could not open raw device

2010-11-25 19:54:16.076: [ default][1189568]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 19:54:16.231: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.232: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.265: [  OCRRAW][1189568]ibctx:1:ERROR: INVALID FORMAT

2010-11-25 19:54:16.265: [  OCRRAW][1189568]proprinit:problem reading the bootblock or superbloc 22

2010-11-25 19:54:16.646: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.647: [  OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.757: [  OCRRAW][1189568]propriowv: Vote information on disk 0 [/dev/raw/raw1] is adjusted from [0/0] to [1/2]

2010-11-25 19:54:16.762: [  OCRRAW][1189568]propriowv: Vote information on disk 1 [/dev/raw/raw2] is adjusted from [0/0] to [1/2]

2010-11-25 19:54:17.193: [  OCRRAW][1189568]propriniconfig:No 92 configuration

2010-11-25 19:54:17.193: [  OCRAPI][1189568]a_init:6a: Backend init successful

2010-11-25 19:54:18.263: [ OCRCONF][1189568]Initialized DATABASE keys in OCR

2010-11-25 19:54:18.478: [ OCRCONF][1189568]csetskgfrblock0: clsfmt returned with error [4].

2010-11-25 19:54:18.478: [ OCRCONF][1189568]Failure in setting block0 [-1]

2010-11-25 19:54:18.478: [ OCRCONF][1189568]OCR block 0 is not set !

2010-11-25 19:54:18.479: [ OCRCONF][1189568]Exiting [status=failed]...

這個是Multipath 的bug 問題，就是我們遇到的第一種情況：

（1）用clsfmt檢查raw 裝置

[root@rac1 bin]# ./clsfmt.bin ocr /dev/raw/raw1

clsfmt: Failure trying to resize OCR file from 205567488 bytes to 205570048 bytes

clsfmt: Received unexpected error 4 from skgfifi

skgfifi: Additional information: -2

Additional information: 205567488

這裡沒有成功， 就是bug。

（2）用Pathch中的新檔案替換clsfmt.bin 檔案。

       Patch 是：p4679769。 這個需要Oracle 帳号才能下載下傳。 是以我将Linux 平台下32位和64位的path都上傳到csdn的下載下傳裡了。 有遇到相同問題的朋友自己去下載下傳。 下載下傳位址：

（3）再次測試：

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw1

clsfmt: successfully initialized file /dev/raw/raw1

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw2

clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes

clsfmt: successfully initialized file /dev/raw/raw2

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw3

clsfmt: successfully initialized file /dev/raw/raw3

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw4

clsfmt: successfully initialized file /dev/raw/raw4

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw5

clsfmt: successfully initialized file /dev/raw/raw5

bug 修複成功了。

（4）清空raw裝置

因為我們已經執行過root.sh 指令，會有相關資訊寫到raw裝置裡。 用dd指令清空這些資訊。 最好留一點空間。 因為在Solaris平台上有預留 1M 空間的做法，不然會出現問題。

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw1 bs=10M count=10

10+0 records in

10+0 records out

104857600 bytes (105 MB) copied, 17.4534 seconds, 6.0 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw2 bs=10M count=10

104857600 bytes (105 MB) copied, 11.4307 seconds, 9.2 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw3 bs=10M count=10

104857600 bytes (105 MB) copied, 18.4136 seconds, 5.7 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw4 bs=10M count=10

104857600 bytes (105 MB) copied, 17.821 seconds, 5.9 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw5 bs=10M count=10

104857600 bytes (105 MB) copied, 17.3289 seconds, 6.1 MB/s

如果不清空raw 裝置，直接執行root.sh 指令，還是會報錯誤：

[root@rac1 bin]# /u01/app/oracle/product/crs/root.sh

錯誤的log 日志如下：

#cat /u01/app/oracle/product/crs/log/rac1/client/ocrconfig_16159.log

2010-11-25 23:27:25.468: [ OCRCONF][2118112]ocrconfig starts...

2010-11-25 23:27:25.469: [ OCRCONF][2118112]Upgrading OCR data

2010-11-25 23:27:25.558: [  OCRRAW][2118112]propriogid:1: INVALID FORMAT

2010-11-25 23:27:25.559: [  OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)

2010-11-25 23:27:25.559: [  OCRRAW][2118112]proprinit: Could not open raw device

2010-11-25 23:27:25.560: [ default][2118112]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 23:27:25.560: [ OCRCONF][2118112]Exporting OCR data to [OCRUPGRADEFILE]

2010-11-25 23:27:25.560: [  OCRAPI][2118112]a_init:7!: Backend init unsuccessful : [33]

2010-11-25 23:27:25.560: [ OCRCONF][2118112]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]

2010-11-25 23:27:25.746: [  OCRRAW][2118112]propriogid:1: INVALID FORMAT

2010-11-25 23:27:25.746: [  OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)

2010-11-25 23:27:25.746: [  OCRRAW][2118112]proprinit: Could not open raw device

2010-11-25 23:27:25.747: [ default][2118112]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 23:27:26.112: [  OCRRAW][2118112]propriogid:1: INVALID FORMAT

2010-11-25 23:27:26.112: [  OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)

2010-11-25 23:27:26.112: [  OCRRAW][2118112]proprinit: Could not open raw device

2010-11-25 23:27:26.120: [  OCRAPI][2118112]a_init:6b!: Backend init unsuccessful : [26]

2010-11-25 23:27:26.120: [ OCRCONF][2118112]Failed to initialized OCR context. error:[PROC-26: Error while accessing the physical storage]

2010-11-25 23:27:26.120: [ OCRCONF][2118112]Exiting [status=failed]...

（5）清空raw 裝置後，在執行/u01/app/oracle/product/crs/root.sh 指令

       這次成功執行。