Receiving Clear alerts after a Blackout–Grid Control

Grid Control allows you to set blackouts for targets so that while your performing maintenance you won’t get notified    You can also use them to disable notifications while your working on an issue.  Nothing worse than being paged multiple times while your trying to fix an issue. 


One of our applications has a component which requires a quick nightly bounce schedule via cron.   So I setup a blackout in Grid Control to start 5 minutes before and extend to 5 minutes after the restart.   However, at 3am I received a lovely page letting me know that an alert has cleared:


Subject: EM Alert: Clear:MyApp PROD - Test MyApp Login Page is now up

Target Name=MyApp
Target type=Web Application
Host=
Occurred At=Mar 5, 2012 3:15:00 AM EST
Message=Test MyApp Login Page is now up: MyApp Login Page has status 6 since 03/05/12 03:15:00 till 03/05/12 03:15:00 in America/New_York. Beacon RCPSC Status: 1 from 02/28/12 09:54:31 till 03/05/12 03:17:13 in -05:00. No new severities found after the blackout Metric data found after blackout, using the latest severity Latest severity from the beacon is 15 at 02/29/12 21:13:04 Beacon votes up Beacon Tenzing Status: 1 from 02/28/12 10:20:19 till 03/05/12 03:16:48 in America/New_York. No new severities found after the blackout Metric data found after blackout, using the latest severity Latest severity from the beacon is 15 at 02/29/12 19:26:14 Beacon votes up The final status of the test is UP from 03/05/12 03:15:00 till 03/05/12 03:17:13
Metric=[Test Response] Status


Strange.    Initially I thought that there may have been a system time issue between the servers, or that the restart had taken longer than I expected.   Looking into it tho that was not the case. A search on metalink turns up that there is a bug:

Bug 10210193 WEB APPS SERVICE GENERATE ERROR WHEN BLACKOUT START the Notification rule used has the metric [Test Response] Status which is generating the notification.


The solution is to remove the remove the [Test Response] metric from the rule.    No sleep interruptions the following night!