EM 13c: How to Change Monitoring Credentials For WebLogic Domain Target

We changed WebLogic user password for enforcing security principal, then WebLogic domain targets show down on OEM. Hence we need update the monitoring credentials on OEM for WebLogic domain.

1) Verify the correct Admin Server hostname and password are correct:

EM Console > Middleware Targets > click the WebLogic Domain > WebLogic Domain dropdown menu > choose WebLogic Server Administration Console

Log in with this username/password

and note the Administration Server hostname as indicated in
- the url at the top of the page > https://<AdminServerFQDN>:<port>/console/...
and/or
- under Servers > AdminServer(admin) > Machine column shows Administration Server hostname 

This step is necessary only if this WebLogic Domain is monitored by multiple agents:
2) Put the WebLogic Domain target under blackout:
Monitoring > Blackouts > Create… > > choose “Blackout” (Create)
Choose “Enable Full blackout…”
Add the WebLogic Domain target (select)

3) Modify the password for the WebLogic Server Administr Server using the hostname and password verified in step 1

There are 2 possible ways to navigate to the Monitoring Credentials in EM:

  • Setup (top right) > Security > Monitoring credentials > choose Oracle WebLogic Domain and click “Manage Monitoring Credentials” > choose the Domain and “Set Credentials”
    The is Domain and Node Manager credentials are set in separate places here
    Test and Save configuration

or

  • From the WebLogic Domain’s Home Page > under WebLogic Domain dropdown men > choose Target setup > Monitoring Credentials
    Both the Domain and Node Manager credentials are set on the same page
    Test and Save configuration

This step is necessary only if step 2 was implemented:
4) If Blackout has not already expired,

Monitoring > Blackouts > View by Blackout Name > choose blackout and “Stop”

5) Refresh WebLogic domain.
From WebLogic Domain home page > WebLogic Domain dropdown > choose “Refresh WebLogic Domain”

EM 13c: Updating Monitoring Credentials For WebLogic Domain Target shows “Password at Management Agent does not match error” (Doc ID 2610563.1)

Oracle Data Guard Most Used Commands

Note: Static “_DGMGRL” entries are no longer needed as of Oracle Database 12.1.0.2 and later release in Oracle Data Guard Broker configurations that are managed by Oracle Restart, RAC On Node or RAC as the Broker will use the clusterware to restart an instance. ( Oracle Data Guard Broker and Static Service Registration (Doc ID 1387859.1)

DGMGRL> connect sys@testdb
Password:
Connected to "TESTDB"
Connected as SYSDBA.
DGMGRL> show configuration;

Configuration - dg_testdb

Protection Mode: MaxPerformance
Members:
testdb - Primary database
stestdb - Physical standby database

Fast-Start Failover: Disabled

Configuration Status:
SUCCESS (status updated 20 seconds ago)

Switchover

DGMGRL> switchover to stestdb;
Performing switchover NOW, please wait...
New primary database "stestdb" is opening...
Oracle Clusterware is restarting database "testdb" ...
Connected to "TESTDB"
Switchover succeeded, new primary is "stestdb"

-- After switchover, run 'srvctl config database -d", the database role has been changed by Data Guard Broker automatically.

$ srvctl config database -d STESTDB
Database unique name: STESTDB
Database name: testdb
.
Start options: open
Stop options: immediate
Database role: PRIMARY

...
..
.

$ srvctl config database -d testdb
Database unique name: TESTDB
Database name: TESTDB
.
Start options: mount
Stop options: immediate
Database role: PHYSICAL_STANDBY

Show Lag

DGMGRL>  SHOW CONFIGURATION  lag

Configuration - dg_testdb

Protection Mode: MaxPerformance
Members:
testdb - Primary database
stestdb - Physical standby database
Transport Lag: 0 seconds (computed 0 seconds ago)
Apply Lag: 2 seconds (computed 0 seconds ago)

Fast-Start Failover: Disabled

Configuration Status:
SUCCESS (status updated 28 seconds ago)
DGMGRL> show database "stestdb";

Database - stestdb

Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 0 seconds ago)
Apply Lag: 0 seconds (computed 0 seconds ago)
Average Apply Rate: 322.00 KByte/s
Real Time Query: OFF
Instance(s):
TESTDB1
TESTDB2 (apply instance)

Database Status:
SUCCESS

TRANSPORT-ON / TRANSPORT-OFF

DGMGRL>  show database "testdb";

Database - testdb

Role: PRIMARY
Intended State: TRANSPORT-ON
Instance(s):
TESTDB1
TESTDB2

Database Status:
SUCCESS

DGMGRL> EDIT DATABASE testdb SET STATE='TRANSPORT-OFF';
Succeeded.
DGMGRL> show database "testdb";

Database - testdb

Role: PRIMARY
Intended State: TRANSPORT-OFF
Instance(s):
TESTDB1
TESTDB2

Database Status:
SUCCESS

APPLY-ON / APPLY-OFF

DGMGRL>  show database "stestdb";

Database - stestdb

Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 70 seconds ago)
Apply Lag: 0 seconds (computed 70 seconds ago)
Average Apply Rate: 114.00 KByte/s
Real Time Query: OFF
Instance(s):
TESTDB1
TESTDB2 (apply instance)

Database Status:
SUCCESS

DGMGRL> EDIT DATABASE stestdb SET STATE='APPLY-OFF';
Succeeded.
DGMGRL> show database "stestdb";

Database - stestdb

Role: PHYSICAL STANDBY
Intended State: APPLY-OFF
Transport Lag: 0 seconds (computed 215 seconds ago)
Apply Lag: 0 seconds (computed 215 seconds ago)
Average Apply Rate: (unknown)
Real Time Query: OFF
Instance(s):
TESTDB1
TESTDB2 (apply instance)

Database Status:
SUCCESS

Change apply instance

DGMGRL> edit database  TESTDB set state='apply-off';
Succeeded.

DGMGRL> edit database TESTDB set state='apply-on' with apply instance ='TESTDB1';

Succeeded.

Monitorable (Read-Only) Properties

DGMGRL>   show database verbose testdb;
...
..
.
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
LogXptStatus = '(monitor)'
SendQEntries = '(monitor)'
RecvQEntries = '(monitor)'
...
..
.

DGMGRL> show database testdb 'InconsistentLogXptProps';
INCONSISTENT LOG TRANSPORT PROPERTIES
INSTANCE_NAME STANDBY_NAME PROPERTY_NAME MEMORY_VALUE BROKER_VALUE

Configurable (Changeable) Properties

DGMGRL>  SHOW DATABASE testdb  LogXptMode;
LogXptMode = 'ASYNC'

DGMGRL> EDIT DATABASE testdb SET PROPERTY LogXptMode='SYNC';
Property "logxptmode" updated

DGMGRL> SHOW DATABASE testdb LogXptMode;
LogXptMode = 'SYNC'

Validate database spfile

DGMGRL> validate database  TESTDB  spfile;
This command cannot be used for the primary database.

DGMGRL> validate database STESTDB spfile;
...
..
.

processes:
testdb (PRIMARY) : 3000
stestdb : 6400
...
..
.

Validate static connect identifier

DGMGRL> validate static connect identifier for all
Oracle Clusterware on database "testdb" is available for database restart.

Oracle Clusterware on database "stestdb" is available for database restart.

validate network configuration for all

DGMGRL> validate network configuration for all
Oracle Clusterware on database "testdb" is available for database restart.

Oracle Clusterware on database "stestdb" is available for database restart.

DGMGRL> validate network configuration for all
Connecting to instance "TESTDB2" on database "testdb" ...
Connected to "TESTDB"
Checking connectivity from instance "TESTDB2" on database "testdb" to instance "STESTDB2" on database "stestdb"...
Succeeded.
Checking connectivity from instance "TESTDB2" on database "testdb" to instance "STESTDB" on database "stestdb"...
Succeeded.
Connecting to instance "TESTDB1" on database "testdb" ...
Connected to "TESTDB"
Checking connectivity from instance "TESTDB1" on database "testdb" to instance "STESTDB2" on database "stestdb"...
Succeeded.
Checking connectivity from instance "TESTDB1" on database "testdb" to instance "STESTDB" on database "stestdb"...
Succeeded.
Connecting to instance "STESTDB2" on database "stestdb" ...
Connected to "STESTDB"
Checking connectivity from instance "STESTDB2" on database "stestdb" to instance "TESTDB2" on database "testdb"...
Succeeded.
Checking connectivity from instance "STESTDB2" on database "stestdb" to instance "TESTDB1" on database "testdb"...
Succeeded.
Connecting to instance "STESTDB" on database "stestdb" ...
Connected to "STESTDB"
Checking connectivity from instance "STESTDB" on database "stestdb" to instance "TESTDB2" on database "testdb"...
Succeeded.
Checking connectivity from instance "STESTDB" on database "stestdb" to instance "TESTDB1" on database "testdb"...
Succeeded.

Oracle Clusterware on database "testdb" is available for database restart.

Oracle Clusterware on database "stestdb" is available for database restart.

DGMGRL>

Validate primary and standby database

DGMGRL> validate database testdb

Database Role: Primary database

Ready for Switchover: Yes

Flashback Database Status:
ttestdb: Off

Managed by Clusterware:
testdb: YES

DGMGRL> validate database stestdb

Database Role: Physical standby database
Primary Database: testdb

Ready for Switchover: Yes
Ready for Failover: Yes (Primary Running)

Flashback Database Status:
testdb: Off
stestdb: Off

Managed by Clusterware:
testdb: YES
stestdb: YES

RMAN-06023 Although Backups Are Available 

‘RMAN  RESTORE DATABASE’ command fails with the following errors by using a backup control file:

RMAN-00571: ====================================================       
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ========
RMAN-00571: ====================================================
RMAN-03002: failure of restore command at 21/11/2024 11:09:9
RMAN-06026: some targets not found - aborting restore
....
RMAN-06023: no backup or copy of datafile 3 found to restore
RMAN-06023: no backup or copy of datafile 2 found to restore
RMAN-06023: no backup or copy of datafile 1 found to restore

If you run ‘list backup of database’, we can see the database backups available.

REASON

The problem here is that there are some files in the Flash Recovery Area that belong to different incarnation than the available backups CURRENT incarnation.

If we start a RESTORE database with a BACKUP controlfile and Flash Recovery Area is defined, RMAN execute and implicit crosscheck and catalog of all the objects in the Flash Recovery Area.

SOLOTION

If the offending cataloged files are small, we can move those files to a directory outside the Flash Recovery Area. Then we need to restart the whole restore procedure. It’s necessary to restore controlfile again

OR

Another solution is to temporarily disable use of the flash recovery area for the duration of the restore and recovery commands.

    To disable Flash Recovery Area you need to undefine db_recovery_file_dest:

    #*.db_recovery_file_dest='<directory>'
    #*.db_recovery_file_dest_size=<size>

    Bounce database

    Restore controlfile and then restore/recover database commands.

    If there are some backuppieces or archivelogs in the Flash Recovery Area that need to be cataloged, then it will be necessary to catalog them manually with:  CATALOG BACKUPPIECE  or CATALOG ARCHIVELOG commands.

    How to Connect Oracle GoldenGate Veridata Agent to Autonomous Data Warehouse

    This post demonstrate how to connect the Oracle GoldenGate Veridata to Autonomous Data Warehouse (ADW) and Autonomous Transaction Processing (ATP) by using the Oracle Wallet.

    Downloaded the Wallet_ADWTESTDB.zip from the ADW/ATP console.
    Unzip the Wallet in the location
    $ unzip Wallet_ADWTESTDB.zip

    Archive: Wallet_ADWTESTDB.zip

    inflating: cwallet.sso

    inflating: tnsnames.ora

    inflating: truststore.jks

    inflating: ojdbc.properties

    inflating: sqlnet.ora

    inflating: ewallet.p12
    inflating: keystore.jks
    Add the following into sqlnet.ora
    WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="/home/oracle/cert/Wallet_ADWTESTDB")))

    SSL_SERVER_DN_MATCH=no
    Remove the comment for the following line in ojdbc.properties
    oracle.net.wallet_location=(SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=${TNS_ADMIN})))
    Copy the jdbc jars into Oracle GoldenGate Veridata Agent installed location
    $ ls -ltr /u01/app/oracle/product/veridata23c_agent/agent/drivers/ojdbc8.jar
    -rw-r----- 1 oracle oinstall 4535290 Nov 2 2:20 /u01/app/oracle/product/veridata23c_agent/agent/drivers/ojdbc8.jar
    Point the Veridata Agent to pick the jdbc 18.3 jars with the following entries in agent.properties
    server.jdbcDriver=ojdbc8.jar 
    server.driversLocation = /u01/app/oracle/product/veridata23c_agent/agent/drivers
    Change the database url
    database.url=jdbc:oracle:thin:@adwtestdb_low?TNS_ADMIN=/home/oracle/cert/Wallet_ADWTESTDB
    Start the agent
    $ ./agent.sh stop
    $ ./agent.sh start agent.properties