Some of the disks may be in CLOSED and IGNORED status when “v$asm_disk” is queried after a broken disk on the Exadata storage is replaced. This is because disks are encountering unexpected problems during automatic drop add operations. This can sometimes cause ORA-600 errors related to LMS and unexpected restart of the instance.
When v$asm_disk is queried as follows, we can see that the disk in the 9th slot on the cell node named orclcel03 has changed.
|
1 2 3 4 5 6 7 |
SQL> select MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,OS_MB,FAILGROUP,LABEL,PATH from v$asm_disk; MOUNT_STATUS HEADER_STATUS MODE_STATUS STATE OS_MB FAILGROUP LABEL PATH ----------- --------- ----------- ------ ----- ---------- --------------- ------ CLOSED MEMBER ONLINE NORMAL 31192 ORCLCEL03 RECO_CD_09_ORCLCEL03 o/192.168.10.19/RECO_CD_09_orclcel03 IGNORED MEMBER ONLINE NORMAL 61436 ORCLCEL03 DATA_CD_09_ORCLCEL03 o/192.168.10.19/DATA_CD_09_orclcel03 CLOSED MEMBER ONLINE NORMAL 19022 ORCLCEL03 DBFS_DG_CD_09_ORCLCEL03 o/192.168.10.19/DBFS_DG_CD_09_orclcel03 |
To solve the problem, it is necessary to add the corresponding disks to the ASM disk groups again with the force option.
The problem will be resolved when the disks are added to the relevant disk groups by using the PATH value in the above query.
Processes must be performed by connecting to the ASM instance as SYSASM.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
[oracle@orcldb01 ~]$ sqlplus / as sysasm SQL*Plus: Release 11.2.0.4.0 Production on Thu Apr 6 17:45:39 2017 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> alter diskgroup DATA add disk 'o/192.168.10.19/DATA_CD_09_orclcel03' force rebalance power 10; Diskgroup altered. SQL> alter diskgroup RECO add disk 'o/192.168.10.19/RECO_CD_09_orclcel03' force rebalance power 10; Diskgroup altered. SQL> alter diskgroup DBFS_DG add disk 'o/192.168.10.19/DBFS_DG_CD_09_orclcel03' force rebalance power 10; Diskgroup altered. |
After adding disks, you can see the status of processes from gv$asm_operation. In order for the Rebalance process to take a short time, it is better to increase the power value of the corresponding disk group. In the example above, the power value is set to 10.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
SQL> set lines 1000 SQL> select * from gv$asm_operation; INST_ID GROUP_NUMBER OPERATION STATE POWER ACTUAL SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE ---------- ------------ --------------- ------------ ---------- ---------- ---------- ---------- ---------- ----------- ------------------------------------------------------------------------------------------------------------------------------------ 6 1 REBAL WAIT 10 8 1 REBAL WAIT 10 5 1 REBAL RUN 10 10 312164 312591 4225 0 7 1 REBAL WAIT 10 4 1 REBAL WAIT 10 1 1 REBAL WAIT 10 2 1 REBAL WAIT 10 3 1 REBAL WAIT 10 8 rows selected. |
