The basics of Alerts and Incidents in Zenduty
Alerts and Incidents are the bedrock of Zenduty. Incoming alerts from integration sources create an incident in Zenduty and sends notifications to your team based on your escalation policies.
Every alert coming from an integration source has an entity_id.
An entity_id points to a resource or an entity in the source application. For example, the entity_id of an alert from Jira will be its Ticket ID, an alert from Datadog will be its Alert ID in Datadog, an alert from Grafana will be its Rule ID.
Multiple alerts can have the same entity_id. entity_id is how Zenduty groups multiple alerts about the same problem into one single incident.
Each alert comes with an alert_type. There are six types of alert types:
- critical
- acknowledged
- resolved
- info
- warning
- error
If the alert source is sending an entity_id, and the alert type is either critical, error or warning, then that alert(depending on your integration level settings) will trigger a single incident. Every alert after that with the same entity_id will be attached to the earlier created incident. An acknowledged alert_type from the source will automatically acknowledge an incident and a resolved alert_type will automatically resolve the incident.
For example:
Alert 1: entity_id:database-alpha, alert_type:critical, message:DB is down
The above alert will create a new incident(I). Incident I is now in the triggered state.
Alert 2: entity_id:database-alpha, alert_type:critical, message:DB is still down
The above alert will not create a new incident, but will be appended as an alert to incident I. Incident I will continue to be in the triggered state.
Alert 3: entity_id:database-alpha, alert_type:resolved, message:DB is up
The above alert will automatically resolve the incident I and will also be appended as an alert to incident I.
Alert 4: entity_id:database-alpha, alert_type:critical, message:DB is down
The above alert will create another new incident - I2. Incident I2 is now in the triggered state.