环境:双节点RAC, oracle 11.2.3
客户电话RAC实例2异常,现场查看日志:
实例2:
Fri Aug 25 09:45:16 2017
Received an instance abort message from instance 1Received an instance abort message from instance 1
Please check instance 1 alert and LMON trace files for detail.Please check instance 1 alert and LMON trace files for detail.
LMS0 (ospid: 24510820): terminating the instance due to error 481
System state dump requested by (instance=2, osid=24510820 (LMS0)), summary=[abnormal instance termination].
System State dumped to trace file /oracle/11.2.0/diag/rdbms/ins/ins2/trace/ins2_diag_21561818.trc
Instance terminated by LMS0, pid = 24510820
实例1
Fri Aug 25 09:44:25 2017
IPC Send timeout detected. Sender: ospid 35783054 [oracle@db1 (LMS1)]
Receiver: inst 2 binc 2073329022 ospid 24183072
IPC Send timeout to 2.2 inc 28 for msg type 65518 from opid 14
Fri Aug 25 09:44:27 2017
Communications reconfiguration: instance_number 2
Detected an inconsistent instance membership by instance 1
Evicting instance 2 from cluster
Waiting for instances to leave: 2
Dumping diagnostic data in directory=[cdmp_20170825094516], requested by (instance=2, osid=24510820 (LMS0)), summary=[abnormal instance termination].
Reconfiguration started (old inc 28, new inc 32)
List of instances:
1 (myinst: 1)
查看/oracle/11.2.0/diag/rdbms/gjj/ins2/trace/ins2_diag_21561818.trc
*** 2017-08-25 14:24:35.900
I'm the voting node
Group reconfiguration cleanup
confirm->incar_num 22, rcfgctx->prop_incar 0
Send my bitmap to master 0
kjzgmappropose : incar 0, newmap -
3000000000000000000000000000000000000000000000000000000000000000
kjzgmappropose : rc from psnd : 30
kjzdattdlm: Can not attach to DLM (LMON up=[TRUE], DB mounted=[FALSE]).