You can silence all of your current outages by clicking on the fire icon in the upper right hand corner of the control panel.

This will open up a window that lists all of your current outages. At the top of this window there is a Cancel All Pending AlertsĀ button. If you press this it will prevent any of these outages from sending out any additional alerts.