oracle.sysman.gcagent.task.TaskTimeoutException: task timeout: 120000 MILLISECONDS for oracle_database: ORCL:Response

There following alert has been received with timeout error message “…oracle.sysman.gcagent.task.TaskTimeoutException: task timeout: 120000 MILLISECONDS…”

 Host=oemnode1 
 Target type=Database Instance 
 Target name=OEMPROD
 Incident creation time=13/11/2020 10:09:39 PM AEDT 
 Last updated time=13/11/2020 10:09:39 PM AEDT 
 Message=Metric evaluation error start - oracle.sysman.emSDK.agent.fetchlet.exception.FetchletException: Timeout during collection: oracle.sysman.gcagent.task.TaskTimeoutException: task timeout: 120000 MILLISECONDS for oracle_database:OEMPROD:Response 
 Severity=Critical 
 Incident ID=463520 
 Event count=1 
 Incident Status=New 
 Escalated=No 
 Priority=High 
 Incident owner=SYSMAN
 Incident Acknowledged By Owner=No 
 Categories=Availability 
 ...
..
.

Subscribe to get access

Read more of this content when you subscribe today.

ORA-10561: block type ‘TRANSACTION MANAGED DATA BLOCK’

The standby database alert log has the following errors:

Tue Nov 17 11:43:51 2020
 Errors in file /u01/app/oracle/diag/rdbms/stbytestdb/STBYTESTDB/trace/STBYTESTDB_mrp0_7269.trc:
 ORA-00600: internal error code, arguments: [3020], [3], [115047], [115047], [], [], [], [], [], [], [], []
 ORA-10567: Redo is inconsistent with data block (file# 3, block# 115047, file offset is 942465024 bytes)
 ORA-10564: tablespace SYSAUX
 ORA-01110: data file 3: '/u02/oradata/STBYTESTDB/sysaux01.dbf'
 ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 8316
 Tue Nov 17 11:43:51 2020
 MRP0: Background Media Recovery process shutdown (STBYTESTDB)

SOLUTION

Subscribe to get access

Read more of this content when you subscribe today.

DataPump: UDE-00008 ORA-31626 ORA-39086

The following errors occur when exporting data by using DataPump:

UDE-00008: operation generated ORACLE error 31626
ORA-31626: job does not exist
ORA-39086: cannot retrieve job information
ORA-06512: at "SYS.DBMS_DATAPUMP", line 2772
ORA-06512: at "SYS.DBMS_DATAPUMP", line 3886
ORA-06512: at line 1

Check the log which shows exporting has been successful.

...
..
.
Master table "SYS"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
Dump file set for SYS.SYS_EXPORT_SCHEMA_01 is:
C:\TEMP\TEST.DMP
Job "SYS"."SYS_EXPORT_SCHEMA_01" successfully completed at 12:08:38

So it is a bug, and this error can be ignored.

How to Restrict Parallel Server Processes Running on Specific Oracle RAC Nodes

For this SQL, I want all the parallel processes running on this node or those nodes only.

All Oracle DBA knows, by default, the parallel server processes spawned to execute a SQL statement can operate on any or all Oracle RAC nodes in the cluster.

For big indexes rebuilding , a client wants all the parallel server processes running on specific node(s).

There are two easy ways to achieve this without bouncing database required.

Subscribe to get access

Read more of this content when you subscribe today.

An internal error has occurred at the agent: “java.lang.IllegalStateException: getAgentSubstitutionMap(“cluster.ClusterName”) == null”

While testing a named credential in OEM 13c by following command with errors:

Here “CLU-RACTEST” is the GI cluster name.

$emcli test_named_credential -cred_names="SEC_RACTEST" -target_name="CLU-RACTEST" -target_type="cluster"

An internal error has occurred at the agent: "java.lang.IllegalStateException: getAgentSubstitutionMap("cluster.CLU-RACTEST") == null". Check the agent's error logs for details.

Run “emctl config agent listtargets”, it shows there is no ‘cluster’ target type listed for any of the monitoring agents.

The <AGENT_INST>/sysman/log/gcagent.log showed the below errors.

2020-09-11 14:20:12,572 [631842:5A0FE62C] WARN - target cluster.CLU-RACTEST does not exist
...
..
.
[159979801250001])] ERROR - remote target "cluster.CLU-RACTEST" has NO substitution properties; check that it is properly configured
2020-09-11 14:20:12,573 [631842:HTTP Listener-631842 - /emd/main/ (DispatchRequests OMS.console@17163@oemnode1=>[159979801250001])] ERROR - Critical error:
java.lang.IllegalStateException: getAgentSubstitutionMap("cluster.CLU-RACTEST") == null
at oracle.sysman.gcagent.jobs.JobRequestDriver.substitute(JobRequestDriver.java:2162)
at oracle.sysman.gcagent.jobs.JobRequestDriver.(JobRequestDriver.java:1500)
at oracle.sysman.gcagent.jobs.JobRequestDriver.(JobRequestDriver.java:1529)
at oracle.sysman.gcagent.dispatch.cxl.PerformOperationAction.satisfyRequest(PerformOperationAction.java:130)
...
..
.

SOLUTION

From the OEM Console, go to Cluster homepage -> Target Setup -> Monitoring Configuration.
Click on the Update button (no need to make any other changes).

This will push the cluster target details to the agent side.

Then run “emcli test_named_credential” successfully.

$emcli test_named_credential -cred_names="SEC_RACTEST" -target_name="CLU-RACTEST" -target_type="cluster"

Credentials "SEC_RACTEST:SYSMAN" tested successfully