SYS.SYS$SERVICE_METRICS_TAB Size Grows Big and Affects AQ Performance

SYMPTOMS

1)Table SYS.SYS$SERVICE_METRICS_TAB keeps growing everyday.

SQL> select count(*) from SYS$SERVICE_METRICS_TAB;

COUNT(*)
----------
46946104

SQL> SELECT TRUNC(enq_time), COUNT(*)
  FROM sys$service_metrics_tab
 GROUP BY TRUNC(enq_time)
 ORDER BY 1 ;

TRUNC(ENQ   COUNT(*)
--------- ----------
07-JAN-19      41052
08-JAN-19      63338
09-JAN-19      63096
10-JAN-19      63338
11-JAN-19      62128
12-JAN-19      63338
13-JAN-19      63316
14-JAN-19      63316
...
..
.

2) Oracle background process ora_cjq — Job scheduler coordinator slows and hangs when running below SQL with “enq: TQ – DDL contention” contention.

begin dbms_aqadm_sys.remove_subscriber (
                            queue_name => 'SYS.SYS$SERVICE_METRICS', 
                 subscriber => sys.aq$_agent(:1, NULL, NULL)); end;

3) Oracle background process Qnnn: Queue cleanup processes hang with “library cache pin” contention.

4) oraagent.bin@host program also has “library cache pin” contention”.

5) Manageability Monitor Process MMON background slave process ora_m000 has “library cache lock” contention.

Subscribe to get access

Read more of this content when you subscribe today.

ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi]

There are a lot ORA-00600 [kwqitnmphe:ltbagi] errors in alert log, this error is caused by a QMON slave background process:

*** 2019-10-05 18:43:28.563
*** SESSION ID:(1906.11789) 2019-10-05 18:43:28.563
*** CLIENT ID:() 2019-10-05 18:43:28.563
*** SERVICE NAME:(SYS$BACKGROUND) 2019-10-05 18:43:28.563
*** MODULE NAME:(Streams) 2019-10-05 18:43:28.563
*** ACTION NAME:(QMON Slave) 2019-10-05 18:43:28.563

kwqitmmit during kwqitmmit:time mgr IOT proc: Error 600 in Queue 
                                Table SYS.SYS$SERVICE_METRICS_TAB
kwqicaclcur: Error 600
Cursor Session Number : 1904
Cursor Session Serial : 24687
Cursor Pin Number : 24585
kwqitmmit during kwqitmmit:end proc: Error 600 in Queue Table 
                                        SYS.SYS$SERVICE_METRICS_TAB
kwqitptm: error 600 encountered during serving 10008981

*** 2019-10-05 18:43:28.564
ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi], 
                        [1], [0], [], [], [], [], [], [], [], [], []
OPIRIP: Uncaught error 447. Error stack:
ORA-00447: fatal error in background process
ORA-00600: internal error code, arguments: [kwqitnmphe:ltbagi], [1], 
                             [0], [], [], [], [], [], [], [], [], []

Subscribe to get access

Read more of this content when you subscribe today.