Learning paths / Operate your application / Troubleshooting playbook

Failure gallery: wrong port, crash loop, out of memory, image pull, bad config

Reading · 7 min · Module 5, lesson 2 of 442 min left in this module

Module 5 · Troubleshooting playbookLesson 2 of 4

Goal: Recognise the five common deploy failures from what the status, deploy log and runtime log show.

Key idea

Most failed deploys are one of five faults, and each leaves a different trace. Learn the traces and you can name the fault from the logs in under a minute.

The five, side by side

FaultDeploy logRuntime logFix
Wrong portUnhealthy, then restartsNormal start-up, listening on another portSet the service's Port to the one the app listens on
Crash loopRestarting repeatedlyThe app's last words, often a fatal lineWhat the line says: usually a missing variable or secret
Out of memoryOut of memoryStops mid-start, no errorUse less memory, or a bigger shape
Image can't be usedNothing: refused before it startsNothingCorrect the name or tag, or add registry credentials
Bad configUnhealthy, or one of the aboveDepends on the settingCorrect the setting in Settings

Wrong port

The app starts fine and logs that it's listening, but on a different port from the service's. With no health check configured, ComputeSphere checks that the app accepts connections on the service's port. Nothing answers there, so it never takes traffic.

The clue is the runtime log looking healthy while the deploy log doesn't.

Crash loop

The app exits during start-up, is restarted, exits again, and the deploy log says Restarting repeatedly. The runtime log holds the reason. From the sample you'll use next:

{"level":"fatal","msg":"SIGNING_KEY is not set: the shop can't sign orders without it. Add SIGNING_KEY as a secret variable and redeploy","missing":"SIGNING_KEY"}

Out of memory

The app used more memory than its shape allows, and was stopped. There's no error line, because the app didn't get to write one: the log simply ends, often during something memory-hungry such as loading a cache. The deploy log adds: "The service exceeded its memory allotment and was stopped. Consider a larger shape."

An image that can't be used

Before anything starts, ComputeSphere checks that it can reach the image. A tag that doesn't exist, a typo, or a private image deployed without credentials is refused straight away, and csph prints image not found or not accessible: with the image name. No service starts, so there are no logs.

And Image pull failed?

Image pull failed in the deploy log means the image passed that first check but the pull still failed, for example because registry credentials stopped working. Check the name, tag and credentials (lesson 5.8.1).

Bad config

A setting the app doesn't agree with. A missing secret shows as a crash loop; a value too big shows as out of memory. A health check on a path that doesn't exist shows as Unhealthy while the runtime log fills with requests to that path answered 404.

Check yourself

The runtime log ends with a line saying the app is listening on 8080. The deploy log says Unhealthy. What do you check first?