Learning paths / Cloud foundations / Servers and compute

CPU and memory, and running out of them

Reading · 5 min · Module 2, lesson 2 of 415 min left in this module

Module 2 · Servers and computeLesson 2 of 4

Goal: Tell a CPU shortage from a memory shortage by how an app behaves, and read both on your own computer.

Key idea

Run out of CPU and your app gets slow: work waits its turn. Run out of memory and your app gets stopped: the operating system ends it to protect everything else. Slow and crashing are different problems with different fixes.

CPU: how much work at once

The CPU runs your code. A modern CPU has several cores, and each core works on one task at a time, switching between tasks many times a second so they all seem to run together.

Cloud providers sell CPU in vCPUs: a virtual CPU, usually one core's worth of processing time. Half a vCPU means your app gets about half of one core.

When there's more work than cores, tasks queue. Nothing breaks; every request just takes longer. At 100% CPU a site doesn't fall over, it crawls.

Memory: how much you hold at once

Memory (RAM) holds everything a running program is using: its code, open connections, the data it's working on. It's fast, and it's emptied whenever the program stops.

Memory can't queue the way CPU can. When programs ask for more than the machine has, the system first slows down badly, and then the operating system stops a program to free memory. On Linux this is the out-of-memory killer. In a container with a memory limit, going over the limit gets it stopped straight away. Either way your app disappears mid-request and has to start again.

So a service that restarts under load usually has a memory problem, not a CPU one.

Try it: read your own computer

Find how many CPU cores and how much memory you have, then watch what's using them.

sysctl -n hw.ncpu        # CPU cores the system can use
sysctl -n hw.memsize     # memory, in bytes
top -o cpu               # live view, busiest first; press q to quit

Open a few browser tabs and watch memory climb. Start something heavy, like a build, and watch CPU jump to near 100%, then settle.

Why free memory looks low

Operating systems use spare memory to cache recently read files, because unused memory is wasted memory. That cache is handed back the moment a program needs it. So a machine showing little "free" memory is usually fine; the number to watch is available.

Check yourself

Your API answers every request, but each one gets slower as traffic grows. CPU sits at 100%. What's happening?
Your app keeps disappearing and restarting during busy periods, with no error from your code. What do you check first?

In the docs