How To Recreate The ASM Password File? (A Failure Story Part 3)

Is it possible to get credentials from crs?

I’ve moved my blog from https://insanedba.blogspot.com to https://dincosman.com Please update your bookmarks and follow/subscribe at the new address for all the latest updates and content. More up-to-date content of this post may be available there.

    In the last two posts, I have been talking about the catastrophic situation we experienced, which was triggered by mirrored disk failures. Up until now, we restored OCR config and recreated the lost ASM disk group which was hosting OCR before, but we could not start the CRS on more than one node.

    We got the error "CRS-5019: All OCR locations on ASM disk groups [DATA], and none of these disk groups are mounted". We recreated the asm password file.  (We should have restored it from the offline disk group.)

Recreating it was not enough. There were some missing internal users.

According to "Doc ID 2341753.1, The users used in Flex ASM", CRSUSER__ASM_001 user is needed by crsd and it should have sysasm privilege, so we gave crs what it needed.

We defined the CRSUSER__ASM_001 user's password on our own, but that was not the proper way. It is an internal user which is created at the grid installation part and its password is given internally. 

We used the ocrdump utility to view OCR and OLR contents by writing the content to a file to detect the CRSUSER__ASM_001 user's credential path and retrieved the CRSUSER__ASM_001 user password  according to the "Doc ID 2139591.1, ODA: CRS Could Not Start on Node Due to Invalid ASM Credentials for The "crsuser__asm_001" Clusterware User" and modified it.

After modifying the CRSUSER__ASM_001 user password, CRS started normally. 

We raised IRON MAN again.

We also changed the backup location of the asm password file location and these are the commands we used.

That's the whole story to heal the cluster and asm disk groups, later we configured dataguard databases again.

Hope it helps.