Tag: ROOTCRS_POSTPATCH_UPDATE_OHASD_SERVICE checkpoint has failed

GI Postpatch Command Fails Due To rpm Database Corruption At The OS Level

The following errors occur when running “$ORACLE_HOME/crs/install/rootcrs.sh -postpatch”:

2024-11-09 13:31:20: Executing cmd: /bin/rpm -q sles-release
2024-11-09 13:31:20: Command output:
>  error: rpmdb: BDB0113 Thread/process 128527/140400711669824 failed: BDB1507 Thread died in Berkeley DB library
>  error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
>  error: cannot open Packages index using db5 -  (-30973)
>  error: cannot open Packages database in /var/lib/rpm
>  error: rpmdb: BDB0113 Thread/process 128527/140400711669824 failed: BDB1507 Thread died in Berkeley DB library
>  error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
>  error: cannot open Packages database in /var/lib/rpm
>  package sles-release is not installed
>End Command output
2024-11-09 13:31:20: Check if the startup mechanism systemd is being used
2024-11-09 13:31:20: Executing cmd: /bin/rpm -qf /sbin/init
2024-11-09 13:31:20: Command output:
>  error: rpmdb: BDB0113 Thread/process 128527/140400711669824 failed: BDB1507 Thread died in Berkeley DB library
>  error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
>  error: cannot open Packages index using db5 -  (-30973)
>  error: cannot open Packages database in /var/lib/rpm
>  error: rpmdb: BDB0113 Thread/process 128527/140400711669824 failed: BDB1507 Thread died in Berkeley DB library
>  error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
>  error: cannot open Packages database in /var/lib/rpm
>  error: rpmdb: BDB0113 Thread/process 128527/140400711669824 failed: BDB1507 Thread died in Berkeley DB library
>  error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
>  error: cannot open Packages database in /var/lib/rpm
>  file /sbin/init is not owned by any package
>End Command output
2024-11-09 13:31:20: Executing cmd: /u01/app/19.0.0/grid/bin/clsecho -p has -f clsrsc -m 180 '/bin/rpm -qf /sbin/init' '1'
2024-11-09 13:31:20: Executing cmd: /u01/app/19.0.0/grid/bin/clsecho -p has -f clsrsc -m 180 '/bin/rpm -qf /sbin/init' '1'
2024-11-09 13:31:20: Command output:
>  CLSRSC-180: An error occurred while executing the command '/bin/rpm -qf /sbin/init'
>End Command output
2024-11-09 13:31:20: CLSRSC-180: An error occurred while executing the command '/bin/rpm -qf /sbin/init'
2024-11-09 13:31:20: ###### Begin DIE Stack Trace ######
2024-11-09 13:31:20:     Package         File                 Line Calling
2024-11-09 13:31:20:     --------------- -------------------- ---- ----------
2024-11-09 13:31:20:  1: main            rootcrs.pl            358 crsutils::dietrap
2024-11-09 13:31:20:  2: s_crsutils      s_crsutils.pm        2729 main::__ANON__
2024-11-09 13:31:20:  3: s_crsutils      s_crsutils.pm        2701 s_crsutils::s_checkLinuxInitMethod
2024-11-09 13:31:20:  4: s_crsutils      s_crsutils.pm        3712 s_crsutils::s_is_Linux_Systemd
2024-11-09 13:31:20:  5: oraClusterwareComp::oraohasd oraohasd.pm          1537 s_crsutils::s_copy_afdinit_init
2024-11-09 13:31:20:  6: crspatch        crspatch.pm           730 oraClusterwareComp::oraohasd::update_OHASD
2024-11-09 13:31:20:  7: crspatch        crspatch.pm          1912 crspatch::updateSystemFiles
2024-11-09 13:31:20:  8: crspatch        crspatch.pm          2249 crspatch::performPostPatch
2024-11-09 13:31:20:  9: crspatch        crspatch.pm           526 crspatch::crsPostPatch
2024-11-09 13:31:20: 10: main            rootcrs.pl            371 crspatch::new
2024-11-09 13:31:20: ####### End DIE Stack Trace #######

2024-11-09 13:31:20: ROOTCRS_POSTPATCH_UPDATE_OHASD_SERVICE checkpoint has failed

Solution

OS level RPM database is corrupted.

Recreate RPM database at OS level to resolve this issue:

1.  As root OS user run the following:

# rm -f /var/lib/rpm/__*
# /bin/rpm –rebuilddb
# echo $?

2.  Validate the rpm database, as the root user:

# /bin/rpm -qa | more


3.  If last command listing rpms correctly, then re-run postpatch command or resume your opatchauto patching session:

As root OS user:

cd $GI_HOME/crs/install
For Cluster Grid Infrastructure Install:#./rootcrs.sh -postpatch

For Restart Grid Infrastructure Install:#./roothas.sh -postpatch

Reference

Postpatch Command Fails on Grid Infrastructure Home Due To rpm Database Corruption At The OS Level (Doc ID 2365433.1)