Grafana Alerts to Sms/Phone/Slack/Microsoft-Teams and Incident Escalation With Zenduty

Grafana is one of the most popular open-source visualization tools that can be used on top of a variety of different data stores but is most commonly used together with Graphite, InfluxDB, Prometheus, Elasticsearch, Prometheus, AWS CloudWatch, and many others. Reliability engineers use Grafana is its ability to bring together several data sources together in a unified dashboard and increase the observability of your production systems.

Grafana is also a popular alerting integration source on Zenduty. Teams that monitor a bunch of metrics across hundreds of servers or applications use Zenduty as a way to alert their on-call engineering teams(SMS/Phone/Slack/Microsoft Teams/Push/Email) whenever a Grafana metric crosses a threshold and ensure swift remediation of critical alerts. The following are the steps to set up Grafana with Zenduty. Detailed instructions can be found here.

Step 1: Sign up on Zenduty. Create a team. Within the team, create a service. Navigate to your Service page and add a Grafana integration. Point the service to one of your escalation policies.

Step 2: In the integration configuration page, copy the webhook URL.

Step 3: Go to your Grafana web page and navigate to “Notification Channels”. Add a new channel. Select type as “Webhook” and give it the name “Zenduty”. Paste the webhook URL previously copied in step 2 and save the notification channel.

Step 4: Go to the graph which you want to monitor for threshold violations, click on “Edit”. In the Alerts tab, setup the configuration for when an alert should be fired.

Step 5: In the “Notifications” tab, add Zenduty as a notification method.

Whenever aa Grafana metric crosses a threshold, Zenduty will trigger an incident and execute the escalation policy and notify the relevant engineers by their set notification channels.

You can also send the Grafana alerts to Slack and Teams channels and manage your Grafana incident end-to-end within your favorite communications channels.