Skip to content

Probes

There are three types of probes that can be applied in a pod manifest:

  • liveness;
  • readiness;
  • startup.

Liveness Probe

This is a probe that allows you to understand whether the application inside the pod is still running. If the probe fails, the container will be restarted according to its restart policy. This probe helps solve deadlock issues and other problems that cause application outages.

The probe can check:

  • the exit status of a bash command and specify the command itself;
  • the status of an HTTP call;
  • TCP Socket (kubelet will try to open a socket to the container on the specified port).

💡 You can read about other checks in "Define a gRPC liveness probe".

In practice, most often we will use some endpoint in our application's API, which Kubernetes will call at a certain frequency. If there is no response from this endpoint or the response has an unexpected status, then kubelet will decide to restart the pod. The liveness probe is called throughout the entire life of the container.

Readiness Probe

This probe is used to determine whether the pod is ready to accept traffic. While the container is NotReady, no traffic from services will be directed to it (we will talk about them in the next topic). Thanks to the readiness probe, we can be sure that traffic will only go to those pods that are ready to accept it and that we won't lose data because of this. Here you can also configure different types of this probe, and in practice it is most often also an endpoint that Kubernetes will call.

Startup Probe

The last probe is Startup, which is used to check whether the application inside the container has already started. If the application has a startup probe, it automatically disables the other two probes until the startup probe succeeds. This probe is needed for applications that require more time to start up and begin working.

Example

Now let's move to practice. We will modify our manifest and add readiness and liveness probes to it. We will also add the corresponding endpoints to our Python code. Let's start by adding the Liveness Probe:

apiVersion: v1
kind: Pod
metadata:
 name: kube2py
 labels:
   app: kube2py
spec:
 containers:
 - name: kube2py
   image: kulyk404/kub2py
   ports:
   - containerPort: 8080
   livenessProbe:
     httpGet:
       path: /health
       port: 8080
     initialDelaySeconds: 10
     periodSeconds: 5

The added livenessProbe section contains the following fields:

  • httpGet — the probe method;
  • /health — the path for the httpGet call;
  • 8080 — the port to which we will make the call;
  • 60 — the initial delay after which requests start being sent (in seconds);
  • 5 — the interval between sending requests (in seconds).

Now let's do the same for the Readiness Probe:

apiVersion: v1
kind: Pod
metadata:
 name: kube2py
 labels:
   app: kube2py
spec:
 containers:
 - name: kube2py
   image: kulyk404/kub2py
   ports:
   - containerPort: 8080
   livenessProbe:
     httpGet:
       path: /health
       port: 8080
     initialDelaySeconds: 60
     periodSeconds: 5
   readinessProbe:
     httpGet:
       path: /ready
       port: 8080
     initialDelaySeconds: 5
     periodSeconds: 5

The added readinessProbe section contains:

  • httpGet — the probe method;
  • /ready — the path for the httpGet call;
  • 8080 — the port to which we will send the request;
  • 5 — the initial delay after which requests start being sent (in seconds);
  • 5 — the interval between sending requests (in seconds).

Let's add two endpoints /health and /ready:

from flask import Flask, Response, send_file
import os
import time

app = Flask(__name__)

start_time = time.time()
# Define the startup period (in seconds)
startup_period = 40

@app.route('/')
def hello():
   return f'''
   Docker is Awesome!
<pre>                   ##        .</pre>
<pre>             ## ## ##       ==</pre>
<pre>          ## ## ## ##      ===</pre>
<pre>      /""""""""""""""""\___/ ===</pre>
<pre> ~~~ (~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===-- ~~~</pre>
<pre>      \______ o          __/</pre>
<pre>        \    \        __/</pre>
<pre>         \____\______/</pre>
   '''

@app.route('/logo')
def docker_logo():
   return send_file('docker-logo.png', mimetype='image/png')

@app.route('/health')
def health_check():
   return Response("Healthy", status=200)

@app.route('/ready')
def readiness_check():
   # You can add logic here to determine if the app is ready. For simplicity, we'll just return 200.
   if time.time() < start_time + startup_period:
       # Application is not ready, return a non-200 status code
       return Response("Not Ready", status=503)
   else:
       # Application is ready, return 200
       return Response("Ready", status=200)

if __name__ == '__main__':
   app.run(host='0.0.0.0', port=8080)

For health, the Response will return Healthy with status 200:

@app.route('/health')
def health_check():
   return Response("Healthy", status=200)

For ready, a timer is set:

start_time = time.time()
# Define the startup period (in seconds)
startup_period = 40

If the application is not ready, status code 503 will be returned as the Response:

@app.route('/ready')
def readiness_check():
   # You can add logic here to determine if the app is ready. For simplicity, we'll just return 200.
   if time.time() < start_time + startup_period:
       # Application is not ready, return a non-200 status code
       return Response("Not Ready", status=503)
   else:
       # Application is ready, return 200
       return Response("Ready", status=200)

Now let's delete our old pod and create a new one. But first, we need to rebuild the image, tag it, push it, and only then create the new one:

docker build . -t ikulyk404/kub2py:1.1.0
docker push ikulyk404/kub2py:1.1.0
kubectl delete pod kube2py -n mateapp
kubectl apply -f app-pod.yml
pod/kube2py created

Let's check all pods:

kubectl get pods -n mateapp
NAME      READY   STATUS    RESTARTS   AGE
busybox   1/1     Running   189        7d21h
kube2py   0/1     Running   0          76s
nginx     1/1     Running   0          7d21h

The pod was created, but the container inside it is not Ready yet — this is because of the delay we set for the readiness probe. Let's look at the logs:

kubectl logs kube2py -n mateapp
 * Serving Flask app 'app'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:8080
 * Running on http://10.1.0.11:8080
Press CTRL+C to quit
10.1.0.1 - - [30/Jan/2024 10:07:14] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:19] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:24] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:29] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:34] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:39] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:44] "GET /ready HTTP/1.1" 503 -
10.1.0.1 - - [30/Jan/2024 10:07:49] "GET /ready HTTP/1.1" 200 -
10.1.0.1 - - [30/Jan/2024 10:07:54] "GET /ready HTTP/1.1" 200 -

kubelet made seven unsuccessful requests before our logic started returning successful statuses.

Another important command is kubectl get events -A. It shows events in a namespace, and the -A flag means the command applies to all namespaces:

kubectl get events -n mateapp
2m4s        Normal    Created     pod/kube2py   Created container kube2py
2m4s        Normal    Started     pod/kube2py   Started container kube2py
88s         Warning   Unhealthy   pod/kube2py   Readiness probe failed: HTTP probe failed with statuscode: 503

Let's verify everything works using port-forward:

kubectl port-forward pod/kube2py 8081:8080 -n mateapp
Forwarding from 127.0.0.1:8081 -> 8080
Forwarding from [::1]:8081 -> 8080