Friday

Grid Control 11g–Blocked Agents

Today I noticed that one of my Grid Control agents was down. From the Grid Control homepage for the target:

clip_image002

I thought it had just crashed so I checked the status from the server:

[oracle@myserver bin]$ ./emctl status agent
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version : 11.1.0.1.0
OMS Version : 11.1.0.1.0
Protocol Version : 11.1.0.0.0
Agent Home : /u01/app/oracle/product/grid11g/agent11g
Agent binaries : /u01/app/oracle/product/grid11g/agent11g
Agent Process ID : 29687
Parent Process ID : 29661
Agent URL : https://myserver:3872/emd/main/
Repository URL : https://myserver:4900/em/upload
Started at : 2011-06-23 02:06:56
Started by user : oracle
Last Reload : 2011-06-23 02:06:56
Last successful upload : (none)
Last attempted upload : (none)
Total Megabytes of XML files uploaded so far : 0.00
Number of XML files pending upload : 432
Size of XML files pending upload(MB) : 52.54
Available disk space on upload filesystem : 2.74%
Collection Status : Disabled by Upload Manager
Last attempted heartbeat to OMS : 2011-06-24 10:37:34
Last successful heartbeat to OMS : unknown
---------------------------------------------------------------
Agent is Running and Ready



Notice the “Last successful heartbeat to OMS” is unknown. If I try and force an upload it errors:

[oracle@myserver bin]$ ./emctl upload
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
EMD upload error: uploadXMLFiles skipped :: OMS version not checked yet. If this issue persists check trace files for ping to OMS related errors.



Checking the agent logs ($GRID_HOME/agent11g/sysman/log/emagent.trc):

2011-06-24 10:40:03,074 Thread-1091160384 ERROR pingManager: Did not receive valid response to ping "ERROR-Agent is blocked. Blocked reason is: Agent is out-of-sync with repository. This most likely means that the agent was reinstalled or recovered. Please contact an EM administrator to unblock the agent by performing an agent resync from the console. Please contact EM adminstrator to unblock the agent"


So the agent is out of sync with with the repository and is now blocked. I remember back in the 9i days to clean up an agent was a bit of a pain but in 11g its pretty simple. Log back into Grid Control, click on the Setup link located in the top right hand corner. Next click on the Agents link in the blue bar, followed by the Misconfiguration Alerts and Blocked Agents tab.

clip_image004

If you click on the agent itself (instead of selecting it and clicking unblock) you will be brought to the agents page shown below:

clip_image006

To resynch it click on the Agent Resynchronization button.

clip_image008

Make sure the Unblock agent on successful completion… is selected and hit continue. Once completed you may notice that the agent is still showing as unreachable. If you check via the server again you’ll notice that its working fine.  Notice the Last successful heartbeat and last successful upload rows.

[oracle@myserver bin]$ ./emctl status agent
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version : 11.1.0.1.0
OMS Version : 11.1.0.1.0
Protocol Version : 11.1.0.0.0
.
.
Last successful upload : 2011-06-24 10:44:36
Total Megabytes of XML files uploaded so far : 19.37
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0.00
Available disk space on upload filesystem : 2.76%
Last successful heartbeat to OMS : 2011-06-24 10:44:04
---------------------------------------------------------------
Agent is Running and Ready



After a short period of time it will show as up in the Grid Control Console as well.

clip_image010

4 comments:

Kelloggsville said...

this helped a lot thanks

Anonymous said...

Great help! Thanks.

Prologic Corporation said...

This is a good article & good site.Thank you for sharing this article. It is help us following categorize:
healthcare, e commerce, programming, multi platform,inventory management, cloud-based solutions, it consulting, retail, manufacturing, CRM, technology means, digital supply chain management, Delivering high-quality service for your business applications,
Solutions for all Industries,
Getting your applications talking is the key to better business processes,
Rapid web services solutions for real business problems,
Web-based Corporate Document Management System,
Outsourcing Solution,
Financial and Operations Business Intelligence Solution,

Our address:
2002 Timberloch Place, Suite 200
The Woodlands, TX 77380
281-364-1799

prologic-corp

Sridevi Koduru said...

Regards
Sridevi Koduru (Senior Oracle Apps Trainer Oracleappstechnical.com)
LinkedIn profile - https://in.linkedin.com/in/sridevi-koduru-9b876a8b
Please Contact for One to One Online Training on Oracle Apps Technical, Financials, SCM, Oracle Manufacturing, OAF, ADF, SQL, PL/SQL, D2K at sridevikoduru@oracleappstechnical.com | +91 - 9581017828.