$ cluvfy comp clocksync -n all -verbose
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP configuration file "/etc/ntp.conf" existence check passed
Checking daemon liveness...
Check: Liveness for "ntpd"
Node Name Running?
---------- ------------------------
racnode2 no
racnode1 no
PRVF-7590 : "ntpd" is not running on node "racnode2"
PRVF-7590 : "ntpd" is not running on node "racnode1"
PRVG-1024 : The NTP Daemon or Service was not running on any of the
cluster nodes.
PRVF-5415 : Check to see if NTP daemon or service is running failed
Result: Clock synchronization check using Network Time Protocol(NTP)
failed
PRVF-9652 : Cluster Time Synchronization Services check failed
Verification of Clock Synchronization across the cluster nodes was
unsuccessful on all the specified nodes.
But ntpd demon process is running :
#systemctl status ntpd ● ntpd.service - Network Time Service Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled; vendor preset: disabled) Active: active (running) since Sun 2019-09-08 21:06:46 AEST; 58min ago Process: 2755 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS) Main PID: 2756 (ntpd) CGroup: /system.slice/ntpd.service └─2756 /usr/sbin/ntpd -u ntp:ntp -g
Debug “cluvfy” or “runcluvfy.sh”:
$ rm -rf /tmp/cvutrace $ mkdir /tmp/cvutrace $ export CV_TRACELOC=/tmp/cvutrace $ export SRVM_TRACE=true $ export SRVM_TRACE_LEVEL=1 $ cluvfy comp clocksync -n all -verbose $ ls -ltr /tmp/cvutrace total 1960 -rw-r--r-- 1 grid oinstall 0 Sep 8 21:46 cvutrace.log.0.lck -rw-r--r-- 1 grid oinstall 0 Sep 8 21:47 cvuhelper.log.0.lck -rw-r--r-- 1 grid oinstall 1586 Sep 8 21:47 cvuhelper.log.0 -rw-r--r-- 1 grid oinstall 2000962 Sep 8 21:47 cvutrace.log.0
From the trace file, it complaints “file check failed” for file “/var/run/ntpd.pid”.
$]$ tail -20 /tmp/cvutrace/cvutrace.log.0
[main] [ 2019-09-08 21:47:03.312 EST ] [VerificationLogData.traceLogData:251] FINE: [Task.perform:514]
m_nodeList='racnode2,racnode1'
[main] [ 2019-09-08 21:47:03.312 EST ] [VerificationLogData.traceLogData:251] INFO: [sVerificationUtil.getUniqueDistributionID:494] DistributionID[0]:7.2
[main] [ 2019-09-08 21:47:03.312 EST ] [VerificationLogData.traceLogData:251] INFO: [sVerificationUtil.getUniqueDistributionID:559] ==== Distribution Id determined to be OL7
[main] [ 2019-09-08 21:47:03.312 EST ] [VerificationLogData.traceLogData:251] FINE: [VerificationCommand.execute:297]
Output: '<CV_VRES>1</CV_VRES><CV_LOG>Exectask: file check failed</CV_LOG><CV_CMDLOG><CV_INITCMD>/tmp/CVU_12.1.0.2.0_grid/exectask -chkfile /var/run/ntpd.pid </CV_INITCMD><CV_CMD>access() /var/run/ntpd.pid F_OK</CV_CMD><CV_CMDOUT></CV_CMDOUT><CV_CMDSTAT>2</CV_CMDSTAT></CV_CMDLOG><CV_ERES>0</CV_ERES>'
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] FINE: [VerificationCommand.execute:297]
Output: '<CV_VRES>1</CV_VRES><CV_LOG>Exectask: file check failed</CV_LOG><CV_CMDLOG><CV_INITCMD>/tmp/CVU_12.1.0.2.0_grid/exectask -chkfile /var/run/ntpd.pid </CV_INITCMD><CV_CMD>access() /var/run/ntpd.pid F_OK</CV_CMD><CV_CMDOUT></CV_CMDOUT><CV_CMDSTAT>2</CV_CMDSTAT></CV_CMDLOG><CV_ERES>0</CV_ERES>'
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] ERROR: [Result.addErrorDescription:624] PRVF-7590 : "ntpd" is not running on node "racnode2"
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] ERROR: [Result.addErrorDescription:624] PRVF-7590 : "ntpd" is not running on node "racnode1"
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] FINE: [Task.perform:594]
TaskDaemonLiveliness:Daemon Liveness[TASKDAEMONLIVELINESS]:TASK_SUMMARY:FAILED:CRITICAL:VERIFICATION_FAILED
ERRORMSG(racnode2): PRVF-7590 : "ntpd" is not running on node "racnode2"
ERRORMSG(racnode1): PRVF-7590 : "ntpd" is not running on node "racnode1"
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] ERROR: [ResultSet.addErrorDescription:1102] PRVG-1024 : The NTP Daemon or Service was not running on any of the cluster nodes.
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] ERROR: [ResultSet.addErrorDescription:1102] PRVF-5415 : Check to see if NTP daemon or service is running failed
[main] [ 2019-09-08 21:47:03.313 EST ] [VerificationLogData.traceLogData:251] FINE: [Task.perform:594]
TaskCTSSIntegrity:Clock Synchronization[TASKCTSSINTEGRITY]:TASK_SUMMARY:FAILED:CRITICAL:VERIFICATION_FAILED
ERRORMSG(GLOBAL): PRVF-5415 : Check to see if NTP daemon or service is running failed
[main] [ 2019-09-08 21:47:03.313 EST ] [CluvfyDriver.main:360] ==== cluvfy exiting normally.
SOLUTION
As per Oracle GI installation documentation, configure ntpd service to start with a pidfile . Edit “/etc/sysconfig/ntpd” and modify the below line
OPTIONS="-g"
to
OPTIONS="-g -p /var/run/ntpd.pid"
# systemctl restart ntpd # ls -l /var/run/ntpd* -rw-r--r-- 1 root root 4 Sep 8 22:21 /var/run/ntpd.pid
$ cluvfy comp clocksync -n all -verbose
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Oracle Clusterware is installed on all nodes.
Checking if CTSS Resource is running on all nodes...
Check: CTSS Resource running on all nodes
Node Name Status
--------- ------------------------
racnode1 passed
racnode2 passed
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
Check: CTSS state
Node Name State
------------ ------------------------
racnode2 Observer
racnode1 Observer
CTSS is in Observer state. Switching over to clock synchronization
checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)
Checking existence of NTP configuration file "/etc/ntp.conf" across
nodes
Node Name File exists?
--------- -----------------------
racnode2 yes
racnode1 yes
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP configuration file "/etc/ntp.conf" existence check passed
Checking daemon liveness...
Check: Liveness for "ntpd"
Node Name Running?
---------- ------------------------
racnode2 yes
racnode1 yes
Result: Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
Checking whether NTP daemon or service is using UDP port 123
on all nodes
Check for NTP daemon or service using UDP port 123
Node Name Port Open?
---------- -----------------------
racnode2 yes
racnode1 yes
Check for synchronization of NTP daemon with at least one external
time source passed on all nodes.
Result: Clock synchronization check using Network Time Protocol(NTP)
passed
Oracle Cluster Time Synchronization Services check passed
Verification of Clock Synchronization across the cluster nodes was
successful.
Make DBA Life Easy 