“ORA-15137: The ASM cluster is in rolling patch state” after applied 19c GI RU

PROBLEM

Just after applied the latest 19c GI July 2020 RU, any operations on diskgroups will cause the following errors :

ORA-15032: not all alterations performed
ORA-15137: The ASM cluster is in rolling patch state.

Subscribe to get access

Read more of this content when you subscribe today.

Check the cluster status back to Normal

-racnode1

[grid@racnode1 ~]$ kfod op=patches
List of Patches
31281355
31304218
31305087
31335188
[grid@racnode1 ~]$
ASMCMD> showclusterstate
Normal
ASMCMD>
[grid@racnode1 ~]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node racnode1 is [441346801].
[grid@racnode1 ~]$
[grid@racnode1 ~]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [441346801] and the complete list of patches [31281355 31304218 31305087 31335188 ] have been applied on the local node. The release patch string is [19.8.0.0.0].
[grid@racnode1 ~]$

-racnode2

[grid@racnode2 ~]$ kfod op=patches
List of Patches
31281355
31304218
31305087
31335188
[grid@racnode2 ~]$
ASMCMD> showclusterstate
Normal
ASMCMD>
[grid@racnode2 ~]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node racnode2 is [441346801].
[grid@racnode2 ~]$
[grid@racnode2 ~]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [441346801] and the complete list of patches [31281355 31304218 31305087 31335188 ] have been applied on the local node. The release patch string is [19.8.0.0.0].
[grid@racnode2 ~]$

opatchauto : Unable to create patchObject

SYMPTOMS

When applying 19c GI JULY 2020 RU ( 19.8.0.0), opatchauto failed with following errors:

#opatchauto apply /tmp/31305339/
...
..
.

Patch: /tmp/31305339/31281355
Log:
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
ORACLE_HOME/inventory/oneoffs/31281355 is corrupted. PatchObject constructor: Input file "/u01/app/19.0.0/grid/inventory/oneoffs/31281355/etc/config/actions" or "/u01/app/19.0.0/grid/inventory/oneoffs/31281355/etc/config/inventory" does not exist.
After fixing the cause of failure Run opatchauto resume
]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.
OPatchauto session completed at Mon Aug 3 21:06:44 2020
Time taken to complete the session 0 minute, 35 seconds
opatchauto failed with error code 42

Subscribe to get access

Read more of this content when you subscribe today.

opatchauto : ContentsXML/oui-patch.xml (Permission denied)

SYMPTOMS

When applying 19c GI JULY 2020 RU ( 19.8.0.0), opatchauto failed with following errors:

[root@racnode2 ~]# opatchauto apply /tmp/31305339/
...
..
.

Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: ApplySession failed in system modification phase… 'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: java.io.FileNotFoundException: /u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)'

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

Subscribe to get access

Read more of this content when you subscribe today.

opatchauto Prerequisite check “CheckActiveFilesAndExecutables” failed

SYMPTOMS

The following errors occur while running opatchauto to apply 19.8 GI RU:

[root@racnode1 ~]# opatchauto apply /tmp/31305339/
...
..
.
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: Prerequisite check "CheckActiveFilesAndExecutables" failed.

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Mon Aug 3 17:50:34 2020
Time taken to complete the session 1 minute, 22 seconds

opatchauto failed with error code 42

Subscribe to get access

Read more of this content when you subscribe today.

How to Apply RU/RUR Patches onto Standalone Database using Fleet Maintenance in Oracle Enterprise Manager Cloud Control

Fleet maintenance uses the Out of Place Patching option with minimum downtime. Also this can be done only via emcli verb and there is no GUI option available.

This post demonstrates how to apply the latest Release Update DB July 2020 Release Update 19.8.0.0.200714   and  OJVM July 2020 Release Update 19.8.0.0.200714 onto standalone database.

Environment:

Primary Host/Database : oemnode1 / OEMREP
Standby Host/Database : stbnode1 / STBOEMREP
Current ORACLE_HOME : /u01/app/oracle/product/19.0.0/dbhome_1
Referenced ORACLE_HOME: /u01/app/oracle/product/19.0.0/dbhome_2
REF_TARGET_NAME : OraDB19Home2_8_oemnode1_8
New Out of Place ORACLE_HOME: /u01/app/oracle/product/19.0.0/dbhome_3

Subscribe to get access

Read more of this content when you subscribe today.