Create alerts with OpenSearch® Dashboards

The OpenSearch® alerting feature sends notifications when data from one or more indices meets certain conditions that can be customized.

Use case examples include monitoring for HTTP status code 503, CPU load average above a certain percentage, or watching for counts of a specific keyword in logs for a specific interval. Notifications can be configured to be sent via email, Slack, or custom webhooks and other channels.

In the following example, we use Slack as the notification channel and a sample-host-health index as the data source to create a simple alert to check CPU load. An action will be triggered when the average of cpu_usage_percentage over 3 minutes is above 75%.

To configure an alert, you need the following:

• Notification channel: a location for notifications to be delivered when an action is triggered
• Monitor: a job that runs on a defined schedule and queries OpenSearch indexes
• Data source: OpenSearch indexes to query
• Query: defines the fields to query from indexes and the way of evaluating results
• Trigger: a defined condition from the query results from the monitor. If a condition is met, the alert is generated.
• Action: a notification configured to be sent through a specified channel when trigger conditions are met. You can define multiple actions.

Create a notification channel

1. In OpenSearch Dashboards, go to Notifications > Channels.

2. Click Create channel.

3. Enter the following:

   1. Name: slack-test

   2. Channel type: Slack

      note

      • Available channel types are: Amazon Chime, Amazon SNS, Slack, Custom webhook, Email, or Microsoft Teams.
      • To use Email, ensure you have an SMTP server configured for a valid domain to deliver email notifications.

   3. Slack webhook URL: Paste your Slack webhook URL.

4. Click Create.

Configure authentication for an email channel

important

Configure authentication for an email channel before configuring the email channel itself.

To authenticate the sender account for sending email messages, add their credentials to the OpenSearch keystore:

1. Go to Aiven Console.

   1. On the Service settings page of your Aiven for OpenSearch service, go to Advanced configuration.

   2. Click Configure > +Add configuration options.

   3. Add all three of the following configuration options and provide the corresponding details for each field:

      • email_sender_name
      • email_sender_username
      • email_sender_password
      note

      Configure all three parameters together. You cannot set them individually or save the configuration with only some of them set.

   4. Click Save configuration.

2. Go to OpenSearch Dashboards.

   1. Go to Notifications > Channels.

   2. Click Create channel.

   3. Enter the following:

      1. Name: email-test
      2. Channel type: Email

   4. Configure a sender:

      1. Sender type: Select SMTP sender.
      2. Select an SMTP sender. If no SMTP sender exists, create one:
         1. Enter a sender name matching the email_sender_name property from the keystore configuration.
         2. Click Create SMTP sender.
         3. Enter the sender details, select Encryption method SSL/TLS, and click Create.

   5. Configure default recipients:

      Select default recipients. If no default recipients exist, create a recipient group:

      1. Click Create recipient group.
      2. Enter the recipient group details, and click Create.

   6. Click Create to save the new channel configuration.

Access Alerting in OpenSearch Dashboards

1. Log in to Aiven Console and go to your Aiven for OpenSearch service.
2. On the service's Overview page, in the Connection information section, go to the OpenSearch Dashboards tab.
3. Open OpenSearch Dashboards by clicking Service URI and logging in.
4. In OpenSearch Dashboards, go to Alerting.

Create a monitor

In OpenSearch Dashboards, go to Alerting > Monitors > Create monitor.

Configure monitor details

In the Monitor details section:

1. Monitor name: Enter High CPU Monitor.
2. Monitor type: Select Per query monitor (selected by default).
3. Monitor defining method: Select Visual editor.
4. Frequency: Select By interval.
5. Run every: Select 1 Minute(s).

note

Frequency can be By interval, Daily, Weekly, Monthly, or Custom CRON expression.

Configure a data source

In the Select data section, configure a data source:

1. Enter sample-host-health as Indexes.
2. Enter timestamp as Time field.

Configure a query

In the Query section, configure a query:

1. Click Add metric.
2. Aggregation: Select average().
3. Field: Select cpu_usage_percentage.
4. Click Save.
5. Time range for the last: Enter 3 minute(s).

Create a trigger

In the Triggers section, create a trigger:

1. Click Add trigger.

2. Trigger name: Enter high_cpu.

3. Severity level: Select 1 (Highest).

4. Trigger condition: Select IS ABOVE and enter 75.

   note

   You can see a visual graph for the trigger with the index data and the defined trigger condition as a red line.

Create an action

In the Triggers section, configure Actions for your trigger.

• To use an existing notification channel for your action:

  1. Action name: Enter slack.
  2. Select your notification channel.
  3. Message subject: Enter High CPU Test Alert.
  4. Enter the message body.

• To use a new notification channel for your action:

  1. Click either Manage channels or Create channels, depending on whether you already have notification channels.
  2. Create a channel.
  3. Return to configuring your action: Go to Alerting > Monitors > Create monitor > Triggers > Actions.
  4. Action name: Enter slack.
  5. Select your new notification channel.
  6. Message subject: Enter High CPU Test Alert.
  7. Enter the message body.

tip

Verify your action configuration by using Preview message and Send test message.

Click Create to finalize your monitor setup.

Related pages

• Alerting monitors configuration
• Notifications plugin