“opatchauto” failed in the middle, try to rerun again, get “CRS-6706” error.
# /u01/app/12.1.0.2/grid/OPatch/opatchauto apply /tmp/12.1.0.2/27010872 -oh /u01/app/12.1.0.2/grid
...
..
.
Using configuration parameter file: /u01/app/12.1.0.2/grid/OPatch/auto/dbtmp/bootstrap_racnode1/patchwork/crs/install/crsconfig_params
CRS-6706: Oracle Clusterware Release patch level ('173535486') does not match Software patch level ('2039526626'). Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.
2018/01/23 16:29:02 CLSRSC-117: Failed to start Oracle Clusterware stack
After fixing the cause of failure Run opatchauto resume
]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.
OPatchauto session completed at Tue Jan 23 16:29:04 2018
Time taken to complete the session 1 minute, 39 seconds
opatchauto failed with error code 42
On racnode1:
$/u01/app/12.1.0.2/grid/bin/kfod op=patches
---------------
List of Patches
===============
19941482
19941477
19694308
19245012
26925218 <---- Does not exist on node2
$/u01/app/12.1.0.2/grid/bin/kfod op=patchlvl
2039526626
On racnode2:
$/u01/app/12.1.0.2/grid/bin/kfod op=patches
---------------
List of Patches
===============
19941482
19941477
19694308
19245012
$/u01/app/12.1.0.2/grid/bin/kfod op=patchlvl
2039526626
We can see patch 26925218 has been applied onto racnode 1, the solution is :
1) Rollback this patch ( 26925218), and run “opatchauto” again to finish the patching successfully. then everything should be fine.
OR
2) Manually complete all the left patches in the GI. After this everything is fine.
In 12c, GI home must have identical patches for the clusterware to start unless during rolling patching.
After applied the same patches on all nodes, GI started fine.
———-
Another situation you might meet is all nodes have same patches but ‘opatch lsinventory’ shows the different patch level:
For example , on racnode1:
$ /u01/app/12.1.0.2/grid/bin/kfod op=patches
---------------
List of Patches
===============
11111111
22222222
33333333
$ /u01/app/12.1.0.2/grid/bin/kfod op=patchlvl
-------------------
Current Patch level
===================
8888888888
Node2
$ /u01/app/12.1.0.2/grid/bin/kfod op=patches
---------------
List of Patches
===============
11111111
22222222
33333333
$ /u01/app/12.1.0.2/grid/bin/kfod op=patchlvl
-------------------
Current Patch level
===================
9999999999
However opatch lsinventory shows the different patch level:
Patch level status of Cluster nodes :
Patching Level Nodes
-------------- -----
8888888888 node1 ====>> different patch level
9999999999 node2
For 12.1.0.1/2:
Execute”/u01/app/12.1.0.2/grid/crs/install/rootcrs.sh -patch” as root user on the problematic node and the patch level should be corrected.
For 12.2
Execute”<GI_HOME>/crs/install/rootcrs.pl -prepatch” “<GI_HOME>/crs/install/rootcrs.pl -postpatch”and as <root_user> on the problematic node and the patch level should be corrected
REFERENCES:
RS-6706: Oracle Clusterware Release patch level (‘nnn’) does not match Software patch level (‘mmm’) (Doc ID 1639285.1)
Make DBA Life Easy 