ComputeSphereLearn
Console
Learning paths / Operate your application / Metrics

Try it: load the sample and read the graphs

Reading · 20 min · Module 3, lesson 3 of 425 min left in this module

Module 3 · MetricsLesson 3 of 4

Goal: Put traffic, CPU load and memory load on a sample service, and explain each curve it draws on the Metrics tab.

3:20 · captions and chapters · narrated with an AI-generated voice
Transcript

Narration uses an AI-generated voice.

[00:00] Where we're going

By the end of this video, you'll put real load on a service, and read what it draws on the Metrics tab. Three graphs. CPU, memory, and spherelets.

[00:11] Set up

Here's the shop, running on Flex. In the first terminal, a loop asks for the products, five times a second. Every line is a two hundred, in well under a second. Leave it running.

In the console, open the service, and choose Metrics. Metrics arrive a few minutes late, so the default hour can look empty. Pick fifteen min, and choose Apply. Now it reads Last fifteen minutes.

[00:38] CPU at the ceiling

In a second terminal, time one heavy request. It does a hundred milliseconds of work, sized for a full processor core. Flex gets a quarter of one, so it took about two hundred and fifty. Write that down.

Now burn CPU for five minutes. It answers straight away, and keeps working in the background. Wait a few minutes, then choose fifteen min and Apply again, to bring the graphs up to now. CPU reads zero point two five vCPU. That's a hundred percent of Flex. The line climbed to the ceiling. It can't go any higher.

[01:17] Slower, not stopped

Time the same request again. Seven hundred and eighty six milliseconds. About three times slower. But look at the first terminal. The product list barely changed. Both wait for CPU. The list needs very little, so the wait hardly shows. Requests that do real work feel a starved CPU first.

[01:39] Memory held on purpose

Stop the burn. Then hold some memory. Two hundred MB. And two hundred more. The app now holds four hundred.

A couple of minutes later, Memory steps up to about four hundred MB, and goes flat. A step that stays flat is memory held on purpose, like a cache. It isn't a leak. And it's still under the plan limit, Flex's five hundred and twelve.

[02:06] Past the ceiling

Now ask for two hundred more. It fails. The app already held four hundred, so it asked for six hundred. That's more than Flex has. The spherelet was stopped, and started fresh. A few seconds later, the next two calls answer, on the new one. And the status shows four hundred again.

To see why, open Logs, then Deploy. Out of memory. The service exceeded its memory allotment, and was stopped. And back on Metrics, the Spherelets graph dips for a moment. The old spherelet stopped, and the new one took its place.

[02:44] Reading the shapes

So, three shapes to remember. CPU flat at the ceiling. The service is starved, and work that needs CPU slows down. Memory that steps up and stays flat. That's memory held on purpose, under the limit. And memory past the limit. The spherelet is stopped and started fresh, and the deploy log says Out of memory. Now it's your turn. Follow the steps in the lesson, and delete the service when you're done.

You need csph pointed at your learn project and lab environment (lesson 5.1.1), room for one more Flex spherelet, and two terminals with curl.

On the free trial?

A trial account runs two Flex spherelets at a time, and shop-api from lab 6.2.4 uses one. If two services are running, stop or delete one that isn't shop-api. This exercise deletes its own service at the end.

  1. Deploy the shop on Flex

    cat > shop-metrics.yaml <<'EOF'
version: "2"
services:
  - name: shop-metrics
    type: web-service
    image: quay.io/computesphere/learn-shop-api:1.4.0
    port: 8080
    plan: flex
    env_vars:
      REQUIRE_SIGNING_KEY: "false"
EOF
csph deploy --file shop-metrics.yaml

    The file is a manifest (lesson 5.8.2). REQUIRE_SIGNING_KEY lets the shop run without the secret from lab 6.2.4; nothing here places orders.

    You should seecsph prints Running and the service's URL.

  2. Start steady traffic

    In the first terminal, with your URL from step 1:

    URL=https://<your-shop-metrics-url>
while true; do curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" "$URL/products"; sleep 0.2; done

    Leave it running. In the console, open shop-metrics, choose Metrics, pick 15 min in the time range, and choose Apply. It then reads Last 15 minutes.

    You should seea new line about five times a second, each 200 and a time well under a second.

  3. Time one heavy request

    In the second terminal:

    URL=https://<your-shop-metrics-url>
curl "$URL/work?ms=100"

    /work does 100 ms of work sized for a full processor core. Flex gets a quarter of one, so it takes longer. Write took_ms down.

    You should see{"took_ms":…,"worked_ms":100}, with took_ms well above 100.

  4. Burn CPU for five minutes

    curl "$URL/burn?seconds=300"

    It answers "burning":true straight away and works in the background. CPU charts lag three to five minutes, so wait, then choose 15 min and Apply again to bring them up to now. The traffic from step 2 barely moved CPU; this fills it to the Flex ceiling.

    You should seewithin five minutes, the CPU chart climbs to about 0.25 vCPU and goes flat.

  5. Time it again

    curl "$URL/work?ms=100"

    Check yourself

    Why did /work slow down so much more than /products?

    You should seetook_ms several times what it was in step 3, while the /products times in the first terminal barely change.

  6. Stop the burn and hold memory

    curl "$URL/burn?seconds=0"
curl "$URL/alloc?mb=200"
curl "$URL/alloc?mb=200"

    A step up that stays flat is memory held on purpose, like a cache. It isn't a leak, and it's still under Flex's 512 MB.

    You should see{"added_mb":200,"alloc_mb":400}, then about two minutes later Memory stepped up to about 400 MB and flat.

  7. Go past the ceiling

    curl "$URL/alloc?mb=200"
curl "$URL/alloc?mb=200"
curl "$URL/alloc?mb=200"
curl "$URL/status"

    The app already held 400 MB, so the first call asked for 600 MB, more than Flex's 512 MB. The spherelet was stopped and started fresh, and calls two and three ran on the new one. The first terminal may show a few errors while it restarts. Open Logs, then Deploy, to see why. For a while the Memory chart can add the old and new spherelets together and spike past 512 MB.

    You should seethe first call fails with a 503 error, the next two answer, /status shows alloc_mb 400, and the deploy log shows Out of memory.

Clean up

Stop the loop in the first terminal with Ctrl-C, then delete the service and the file:

csph services list
csph services delete <shop-metrics-service-id>
rm shop-metrics.yaml

In the docs