Dataguard broker keeps killing RSM process overnight and oncall DBA is pagered by alerts.
DG 2014-10-04-04:17:17 0 2 0 DMON: killing process RSM0, pid = 26742
DG 2014-10-04-04:17:18 0 2 0 DMON: waiting for subscribers to disappear...
DG 2014-10-04-04:17:18 0 2 0 PMON: delete state object for RSM0
DG 2014-10-04-04:17:18 0 2 0 PMON: RSM0 died unexpectedly while processing request 1.1.833218860,
DG 2014-10-04-04:17:18 0 2 0 notifying DMON of RSM failure
DG 2014-10-04-04:17:18 0 2 0 DMON: Creating process RSM0
DG 2014-10-04-04:17:21 0 2 0 RSM0: Attach state object
DG 2014-10-04-04:17:21 0 2 0 DMON: Process RSM0 re-created with pid = 22898
WORKAROUND
DGMGRL> show configuration OperationTimeout;
OperationTimeout = '30'
DGMGRL> EDIT CONFIGURATION SET PROPERTY OperationTimeout=120;
Property "operationtimeout" updated
DGMGRL> show configuration OperationTimeout;
OperationTimeout = '120'
DGMGRL>
Reference Oracle Doc:
Data Guard: Server Hang And Crash Because RSM0 Keeps Re-spawning (Doc ID 1322877.1)
Cause : The Broker is killing the RSM0 process and restarting it because of an internal timeout that is too low.
Make DBA Life Easy 