Learning paths / Operate your application / Monitoring and alerts

Lab: create an alert

Verified lab · 20 min · Module 6, lesson 4 of 525 min left in this module

Module 6 · Monitoring and alertsLesson 4 of 5

Preview. Steps 1 to 5 work today. The alert in step 6 won't reach your inbox yet; delivery is being fixed on the platform.

Goal: Send your notifications to a webhook inbox, create a CPU alert rule for the lab environment, and switch the project's alerts on.

You need csph signed in with learn and lab as defaults (lesson 5.1.1), shop-api Running from lab 6.2.4, and room for one more Flex spherelet.

  1. Deploy the webhook inbox

    csph deploy --image quay.io/computesphere/learn-webhook-inbox:1.4.0 --name webhook-inbox --port 8080
    

    It keeps the last 20 messages sent to it, in memory.

    You should seecsph prints Running and the inbox's URL.

  2. Open your inbox

    Make up a token nobody would guess: 6 to 64 letters, digits, - or _, such as learn-7f3a9c. Open https://<your-inbox-url>/<token> in a browser and keep it open. It refreshes itself.

    You should seeA page headed Inbox, followed by your token, with no messages yet.

  3. Send your notifications to it

    In the console, open Settings, then Notifications. Under Events, turn on Activity alerts. Turn on Webhook, choose Add webhook endpoint, set URL to https://<your-inbox-url>/hooks/<token> and Format to JSON. Choose Save changes.

    You should seeAfter a reload, the endpoint is still listed under Webhook.

  4. Create a CPU rule for lab

    Open the learn project, choose ⋯ next to its name, then Alerts, then New alert. Choose CPU, Threshold (%) 80, Evaluation period (seconds) 300, Severity Medium, Environment lab, then Create.

    You should seeA CPU row marked Medium severity for lab, with Threshold 80% and Evaluation period 5 min, switched on.

  5. Switch the project's alerts on

    On the Alerts page, turn on the Alerts on switch and choose Turn on alerts. It appears once a rule exists. Without it, rules notify nobody.

    You should seeThe Alerts on switch in the Alert rules header is on.

  6. Make it fire

    curl "https://<your-shop-api-url>/burn?seconds=420"
    

    The URL is on shop-api's page in the console. This keeps one processor thread busy: all the CPU of a Flex spherelet, the shape shop-api got by default. After five minutes above 80%, the rule fires, and an alert arrives in your inbox once delivery is working.

    You should seeshop-api's Metrics tab shows CPU at its ceiling for about seven minutes.

Check your work

  • If this doesn't pass

    Check the rule's own switch is on, and that its environment is lab, not another one in the project.

  • If this doesn't pass

    Check the URL ends in /hooks/ and your token, and that you chose Save changes.

  • If this doesn't pass

    Open /<token>, not /hooks/<token>, and wait for webhook-inbox to be Running. Messages arrive once alerts are delivered.

Clean up, once your checks pass
  1. Delete the rule: the bin icon on its row, then Delete (or csph alerts delete <alert-id>). Rules aren't deleted with their project, so always delete them first.
  2. In Notifications, choose Remove endpoint, turn Webhook off, and choose Save changes.
  3. Delete the inbox: csph services list, then csph services delete <webhook-inbox-service-id>.