ORA-15137 ASM Cluster in “rolling patch mode” new disk can’t be rebalanced

What happened?

We patch our cluster with the a PSU but we did not recognize that the cluster was afterwards in rolling patch state. A few days later we had a defect disk in one cell server. The disk was changed and we got an error message.


ASM alert.log

SQL> /* Exadata Auto Mgmt: DROP ASM Disk */
alter diskgroup DBFS_DG drop;
 disk DBFS_DG_CD_08_S1R8CD5;
 force rebalance nowait

ORA-15032: not all alterations performed
ORA-15137: The ASM cluster is in rolling patch state.

So what can be done to get the disk group rebalanced. First we stop the rolling patch mode on the ASM Instance which could be done with the following command

ALTER SYSTEM STOP ROLLING PATCH;

Looking to the Cell Server we saw that the new disk was now known by the Systems but the Grid disks are offline.


CellCLI> list griddisk attributes name,asmmodestatus,asmdeactivationoutcome
 DATA_CD_08_s1r8cd5 OFFLINE Yes
 DBFS_DG_CD_08_s1r8cd5 OFFLINE Yes
 RECO_CD_08_s1r8cd5 OFFLINE Yes

So we saw that the Grid disks are offline

CellCLI> alter griddisk DATA_CD_08_s1r8cd5 inactive;
GridDisk DATA_DG_CD_08_s1r8cd5 successfully altered

CellCLI> alter griddisk DBFS_DG_CD_08_s1r8cd5 inactive;
GridDisk DBFS_DG_CD_08_s1r8cd5 successfully altered

CellCLI> alter griddisk RECO_DG_CD_08_s1r8cd5 active;
GridDisk RECO_DG_CD_08_s1r8cd5 successfully altered

 

While doing these steps the rebalance on the disk groups starts automatically.

ASM alert.log
...
alter diskgroup DBFS_DG drop disk DBFS_DG_CD_08_S1R8CD5
force rebalance nowait;
...
After a few hours the disk groups are in “sync”.

We fixed this problem on the ASM instance but on the Grid Infrastructure level we have a different patchlevel if we do an “opatch lsinv”. To fix this problem I write a second blog article. stay tuned

 

Advertisements

Über spa

Oracle and Unix Professional, main focus on Oracle HA - Systems also an Exadata enthusiasts
Dieser Beitrag wurde unter Exadata, Oracle Database abgelegt und mit , , verschlagwortet. Setze ein Lesezeichen auf den Permalink.

Kommentar verfassen

Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

WordPress.com-Logo

Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

Google+ Foto

Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

Verbinde mit %s