The three probes and what each one does
Kubernetes cannot tell whether your application works; it can only see that a process is running. Probes are how you tell it more. Each probe asks a different question and triggers a different action, and most probe-related outages come from mixing them up.
Startup probe
"Has the application finished starting?" Until it passes, the other probes are held back. It gives slow starters time without slowing failure detection afterwards.
Liveness probe
"Is the process stuck beyond recovery?" On failure the container is restarted. Check only the process itself — never its dependencies.
Readiness probe
"Can this pod take traffic right now?" On failure the pod is removed from the Service but left running. It may check dependencies the pod cannot work without.
Timing fields
periodSeconds × failureThreshold is how long a problem lasts before Kubernetes acts. Shorter is faster, and more likely to act on a blip.
Mistakes that turn probes into outages
A liveness probe that checks the database. When the database slows down, every pod fails liveness and is restarted together. The restarted pods reconnect all at once and make the database slower still. Liveness should answer "is this process alive?" and nothing else.
A timeout tighter than the application under load. The default timeoutSeconds is 1. A health endpoint that takes 1.2 seconds during a traffic peak fails, pods are restarted, the remaining pods carry more load and fail too. Make the health endpoint cheap and give it room.
No startup probe on a slow starter. A liveness probe that begins before the app is ready kills it in a loop — CrashLoopBackOff — especially after a change that makes startup a little slower than before.
Frequently asked questions
What is the difference between liveness and readiness probes?
A failed liveness probe restarts the container; a failed readiness probe only stops traffic being sent to the pod. Use readiness for temporary conditions and liveness for states that only a restart can fix.
When do I need a startup probe?
Whenever startup can take longer than the liveness probe would tolerate — typically more than 20 to 30 seconds. The startup probe gives the application time once, then hands over to the liveness probe.
How long does it take Kubernetes to restart a hung container?
Roughly periodSeconds × failureThreshold plus the timeout. With the common 10-second period and threshold of 3, about 30 seconds after the container stops responding.
Should the liveness probe check dependencies such as the database?
No. If a shared dependency fails, all pods fail liveness at once and are restarted, which does not fix the dependency and removes your whole capacity. Put dependency checks in readiness, if anywhere.