Oracle OEM 13c Metric Collection Error..Or..”Show Me The Graphs!”

When implementing the OEM 13C version, we have a server where we house about 16 databases. Adding the first bunch of the databases went like a breeze, but when all was said and done, we had 3 databases which didn’t want to be monitored..Removing, adding again, all to no avail..

Long story short:

The error message:

Message=Metric evaluation error start – Received an exception when evaluating sev_eval_proc for:Target name = [DBNAME]_SYS, metric_name = Response, metric_column = Status; Error msg = Target encountered metric erros; at least one member in in metric error.

(Yes, there are typos in the error message..for once, their not mine 😉 )

Selection_001

It is not the agent, it is not the (shared) oracle home…it’s the specific database self:

In 13c cloud control, database target status shows metric collection error as below:
“Dynamic Category property error, Get dynamic property error, No such metadata, Computation of a critical dynamic property failed. Retries Completed”

Computing dynamic properties of the target fails with below error:

<AGENT_INST_HOME>/bin>./emctl reload agent dynamicproperties DBNAME:oracle_database
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
EMD reload error:Target oracle_database.orcl is broken: 
Target {oracle_database.orcl} is broken: Dynamic Category property error,
Get dynamic property error,No such metadata - No valid queryDescriptor 
or executionDescriptor found for target [oracle_database.orcl$57] is 
broken: Dynamic Category property error,Get dynamic property error,
 No such metadata - No valid queryDescriptor or executionDescriptor 
found for target [oracle_database.DBNAME$66]

*Note: the dbname is Case Sensitive!

The dynamic properties are these:

There is no bundle_series column in dba_registry_history table for the problematic database as below:

SQL> desc dba_registry_history

Name Null? Type
----------------------------------------- -------- ----------------------------
ACTION_TIME TIMESTAMP(6)
ACTION VARCHAR2(30)
NAMESPACE VARCHAR2(30)
VERSION VARCHAR2(30)
ID NUMBER
COMMENTS VARCHAR2(255)

How to fix?

Fairly easy: just execute catbundle.sql on problematic database to ensure that bundle_series column exists in dba_registry_history table.

Example:

Log in as sys on the database.

SQL*Plus: Release 11.2.0.4.0 Production on Mon May 9 12:32:26 2016
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> @?/rdbms/admin/catbundle.sql
Enter value for 1: cpu
Enter value for 2: apply

PL/SQL procedure successfully completed.
PL/SQL procedure successfully completed.

....<snip>....

SQL> COMMIT;

Commit complete.

SQL> SPOOL off
SQL> SET echo off
Check the following log file for errors:
/oracle/11.2.0.4/base/cfgtoollogs/catbundle/catbundle_CPU_DBNAME_APPLY_2016May09_12_32_44.log

SQL>

Or just check the dba_registry_history table again:

SQL> desc dba_registry_history
Name Null? Type
---------------------------------------------------------------------------
ACTION_TIME TIMESTAMP(6)
ACTION VARCHAR2(30)
NAMESPACE VARCHAR2(30)
VERSION VARCHAR2(30)
ID NUMBER
BUNDLE_SERIES VARCHAR2(30)
COMMENTS VARCHAR2(255)

Tadaa!!

Re-run the metric collection again:

$ ./emctl reload agent dynamicproperties DBNAME:oracle_database
Oracle Enterprise Manager Cloud Control 13c Release 1
Copyright (c) 1996, 2015 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD recompute dynprops completed successfully

Refresh the overview page in OEM, and all should be green..

 

 

Advertisements

About GemsOfProgramming

Beeing a previously enthusiastic Java programmer, I rolled into the Oracle Database Administration world. It turned out I got a knack for this, and since approx. 2000 I'm a full time DBA. My experiences touches lot of Oracle products like Forms and Reports 9/10, JDAPI, Application Server, Weblogic Fusion and of course: Oracle Enterprise Databases, JavaFX, Swing and other Java components.
This entry was posted in Databases, Technical Stuff and tagged , , . Bookmark the permalink.

6 Responses to Oracle OEM 13c Metric Collection Error..Or..”Show Me The Graphs!”

  1. Zola Zimase says:

    Excellent. Thank you very much. This helped me to resolve the same issue.

  2. Juan says:

    Thank you. I’m facing the same issue and Metalink don’t help me the concrete way than your article do.

  3. josh says:

    Hey,
    I’m having the same issue except I do have the bundle_series column in the problematic databases. This error seemed to happen after we applied the 170117 PSU. I have yet to apply to the new April PSU, so maybe that will fix it. 😦

    • Let me know if the update works out for you.. I’m not able to investigate further since I don’t have this setup anymore. However I do have a mega article in the works with the whole upgrade and update process from oracle 11g RAC including ADG..detailed step by step..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s