Kubernetes Deployment and Testing Strategies

Q. What is Kubernetes?

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. Kubernetes automates the deployment, scaling, maintenance, scheduling, and operation of multiple application containers across clusters of nodes. Kubernetes contains tools for orchestration, service discovery, and load balancing that can be used with Docker and Rocket containers. As needs change, a developer can move container workloads in Kubernetes to another cloud provider without changing the code.

Kubernetes Terminology and Architecture

Kubernetes introduces a lot of vocabulary to describe how your application is organized. We’ll start from the smallest layer and work our way up

i. Containers

Containers are software images that represent complete runnable applications .

It differs from a VM as in a VM is an abstraction at the hardware level whereas a container is an abstraction at the app level.A docker container (commonly used ) is a lightweight , standalone executable package of software which includes everything.

1. Code
2. Runtime
3. System tools
4. System libraries
5. Settings

Kubernetes comes into play to orchestrate multiple containers similar to management of a distributed system

ii. Pods

A Kubernetes pod is a group of containers, and is the smallest unit that Kubernetes administers. Pods have a single IP address that is applied to every container within the pod. Containers in a pod share the same resources such as memory and storage. This allows the individual Linux containers inside a pod to be treated collectively as a single application, as if all the containerized processes were running together on the same host in more traditional workloads

iii. Deployments

Kubernetes deployments define the scale at which you want to run your application by letting you set the details of how you would like pods replicated on your Kubernetes nodes.

iv. Services

A service is an abstraction over the pods, and essentially, the only interface the various application consumers interact with. As pods are replaced, their internal names and IPs might change. A service exposes a single machine name or IP address mapped to pods whose underlying names and numbers are unreliable. A service ensures that, to the outside network, everything appears to be unchanged

v.Nodes

A Kubernetes node manages and runs pods; it’s the machine (whether virtualized or physical) that performs the given work.

The Kubernetes control plane

The Kubernetes control plane is the main entry point for administrators and users to manage the various nodes.

The control plane includes

1. API server
2. Scheduler
3. Controller manager

Worker Node components

1. Kubelet- A Kubelet tracks the state of a pod to ensure that all the containers are running. It provides a heartbeat message every few seconds to the control plane.
2. Kube proxy - The Kube proxy routes traffic coming into a node from the service. It forwards requests for work to the correct containers.
3. Etcd - It is a distributed key-value store that Kubernetes uses to share information about the overall state of a cluster.

Kubernetes architecture

A bit more detailed architecture using the above mentioned terminologies

Deployment in Kubernetes

Deployments represent a set of multiple, identical Pods with no unique identities. A Deployment runs multiple replicas of your application and automatically replaces any instances that fail or become unresponsive. In this way, Deployments help ensure that one or more instances of your application are available to serve user requests.

Deployment Strategies in Kubernetes

In Kubernetes there are a few different ways to release an application, it is necessary to choose the right strategy to make your infrastructure reliable during an application update. Here are few deployment strategies that are possible to adopt:

1. Recreate

The recreate strategy is a dummy deployment that consists of shutting down version A then deploying version B after version A is turned off. This technique implies downtime of the service that depends on both shutdown and boot duration of the application.

This is the easiest strategy, which means we just nuke everything, and recreate the new version. Usually good for Dev/Test scenarios or very low impact applications We can simplify this in a single term “Destroy the old, provision the new, don’t care for consequences.”.

The Recreate strategy will:

1. Execute any pre lifecycle hook.
2. Scale down the previous deployment to zero.
3. Execute any mid lifecycle hook.
4. Scale up the new deployment.
5. Execute any post lifecycle hook.

During scale-up, if the replica count of the deployment is greater than one, the first replica of the deployment will be validated for readiness before fully scaling up the deployment. If the validation of the first replica fails, the deployment will be considered a failure.

When to Use a Recreate Deployment

• When you must run migrations or other data transformations before your new code starts.
• When you do not support having new and old versions of your application code running at the same time.
• When you want to use a RWO volume, which is not supported being shared between multiple replicas.

Pros:

• Easy to set up.
• Application state entirely renewed

Cons:

• High impact on the user, expect downtime that depends on both shutdown and boot duration of the application.

2. Rolling Update

Q.What is a rolling update?

All users expect applications to be available all the time and developers are expected to deploy new versions of them many times a day. This is where rolling updates are used in Kubernetes. Rolling updates allow Deployment Updates to take place with zero downtime. They do this by incrementally updating Pods instances with new ones. The new Pods will be scheduled on Nodes with available resources.

If a deployment is exposed publicly, the service will load-balance the traffic only to available Pods during the update. An available Pod is an instance that is available to the users of the application.

Rolling updates and their actions:

• Promoting an application from one environment to another
• Rollback to previous versions
• Continuous Delivery and Integration of applications with zero downtime

Unlike in blue-green deployment which has two complete environments, rolling update hasten complete environments.

In rolling update, new pods are added gradually and old pods are terminated gradually.

In most cases, RollingUpdate is the preferable update strategy for Deployments. Recreate can be useful if you are running a pod as a singleton, and having a duplicate pod for even a few seconds is not acceptable.

Pros:

• Easy to set up.
• Versions are slowly released across instances.
• Convenient for stateful applications that can handle rebalancing of the data.

Cons:

• Rollout/rollback can take time.
• Supporting multiple APIs is hard.
• No control over traffic.

3. Blue Green Deployment

What is Blue Green Deployment?

With blue/green deployments a new copy of the application (green) is deployed alongside the existing version (blue). Then the ingress/router to the app is updated to switch to the new version (green). You then need to wait for the old (blue) version to finish the requests sent to it, but for the most part traffic to the app changes to the new version all at once.

Kubernetes doesn’t have support for blue/green deployments built in. Currently the best way to do it is create a new deployment and then update the service for the application to point to the new deployment. Let’s look at what that means.

We can create our “blue” deployment by saving the following yaml to a file blue.yaml.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-1.10
spec:
  replicas: 3
  template:
    metadata:
      labels:
        name: nginx
        version: "1.10"
    spec:
      containers: 
        - name: nginx
          image: nginx:1.10
          ports:
            - name: http
              containerPort: 80

You can then create the deployment using the kubectl command.

$ kubectl apply -f blue.yaml

Once we have a deployment we can provide a way to access the instances of the deployment by creating a Service. Services are decoupled from deployments so that means that you don't explicitly point a service at a deployment. What you do instead is specify a label selector which is used to list the pods that make up the service. When using deployments, this is typically set up so that it matches the pods for a deployment.

In this case we have two labels, name=nginx and version=1.10. We will set these as the label selector for the service below. Save this to service.yaml.

apiVersion: v1
kind: Service
metadata: 
  name: nginx
  labels: 
    name: nginx
spec:
  ports:
    - name: http
      port: 80
      targetPort: 80
  selector: 
    name: nginx
    version: "1.10"
  type: LoadBalancer

Creating the service will create a load balancer that is accessible outside the cluster.

$ kubectl apply -f service.yaml

You can test that the service is accessible and get the version.

$ EXTERNAL_IP=$(kubectl get svc nginx -o jsonpath="{.status.loadBalancer.ingress[*].ip}")
$ curl -s http://$EXTERNAL_IP/version | grep nginx

Creating Green Deployment

For the “green” deployment we will deploy a new deployment in parallel wit the “blue” deployment. If the following is in green.yaml...

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-1.11
spec:
  replicas: 3
  template:
    metadata:
      labels:
        name: nginx
        version: "1.11"
    spec:
      containers: 
        - name: nginx
          image: nginx:1.11
          ports:
            - name: http
              containerPort: 80

Deployment command

$ kubectl apply -f green.yaml

Updating the App

To cut over to the “green” deployment we will update the selector for the service. Edit the service.yaml and change the selector version to "1.11". That will make it so that it matches the pods on the "green" deployment.

apiVersion: v1
kind: Service
metadata: 
  name: nginx
  labels: 
    name: nginx
spec:
  ports:
    - name: http
      port: 80
      targetPort: 80
  selector: 
    name: nginx
    version: "1.11"
  type: LoadBalancer

This apply will update the existing nginx service in place.

$ kubectl apply -f service.yaml

Updating the selector for the service is applied immediately and so you should see that the new version of nginx is serving traffic.

$ EXTERNAL_IP=$(kubectl get svc nginx -o jsonpath="{.status.loadBalancer.ingress[*].ip}")
$ curl -s http://$EXTERNAL_IP/version | grep nginx

Automating

You can automate your blue/green deployment a bit with some scripting.

#!/bin/bash

# bg-deploy.sh <servicename> <version> <green-deployment.yaml>
# Deployment name should be <service>-<version>

DEPLOYMENTNAME=$1-$2
SERVICE=$1
VERSION=$2
DEPLOYMENTFILE=$3

kubectl apply -f $DEPLOYMENTFILE

# Wait until the Deployment is ready by checking the MinimumReplicasAvailable condition.
READY=$(kubectl get deploy $DEPLOYMENTNAME -o json | jq '.status.conditions[] | select(.reason == "MinimumReplicasAvailable") | .status' | tr -d '"')
while [[ "$READY" != "True" ]]; do
    READY=$(kubectl get deploy $DEPLOYMENTNAME -o json | jq '.status.conditions[] | select(.reason == "MinimumReplicasAvailable") | .status' | tr -d '"')
    sleep 5
done

# Update the service selector with the new version
kubectl patch svc $SERVICE -p "{\"spec\":{\"selector\": {\"name\": \"${SERVICE}\", \"version\": \"${VERSION}\"}}}"

echo "Done."

Pro:

• instant rollout/rollback.
• avoid versioning issue, change the entire cluster state in one go.

Cons:

• requires double the resources.
• proper test of the entire platform should be done before releasing to production.
• handling stateful applications can be hard.

4. Canary deployment/ Canary testing:

Canary deployment is like blue-green, except it’s more risk-averse. Instead of switching from blue to green in one step, you use a phased approach. With canary deployment, you deploy a new application code in a small part of the production infrastructure.

A canary is used when you want to test some new functionality typically on the backend of your application. A canary deployment consists of routing a subset of users to a new functionality. In Kubernetes, a canary deployment can be done using two Deployments with common pod labels. One replica of the new version is released alongside the old version. Then after some time and if no error is detected, scale up the number of replicas of the new version and delete the old deployment.

Using this ReplicaSet technique requires spinning-up as many pods as necessary to get the right percentage of traffic. That said, if you want to send 1% of traffic to version B, you need to have one pod running with version B and 99 pods running with version A.

In the following example, we use two ReplicaSets side by side, version A with three replicas (75% of the traffic), version B with one replica (25% of the traffic).

Canary deployments with a declarative syntax

In a similar manner as the blue/green deployment plugin, the Canary plugin is also taking care of all the kubectl invocations needed behind the scenes. To use it you can simply insert it in a Codefresh pipeline as below:

How to perform Canary deployments

When you run a deployment in Codefresh, the pipeline step will print messages with its progress:

First, the Canary plugin will read the Kubernetes services and extract the “version” metadata label to find out which version is running “in production”. Then it will read the respective deployment and find the Docker image currently getting live traffic. It will also read the number of current replicas for that deployment.

Then it will create a second deployment using the new Docker image tag. This second deployment uses the same labels as the first one, so the existing service will serve BOTH deployments at the same time. A single pod for the new version will be deployed. This pod will instantly get live traffic according to the total number of pods. For example, if you have in production 3 pods and the new version pod is created, it will instantly get 25% of the traffic (1 canary, 3 production version).

Once the first pod is created, the script is running in a loop where each iteration does the following:

• Increases the number of canaries according to the predefined percentage. For example, a percentage of 33% means that 3 phases of canaries will be performed. With 25%, you will see 4 canary iterations and so on. The algorithm used is pretty basic and for a very low number of pods, you will see a lot of rounding happening.
• Waits for some seconds until the pods have time to start (the time is configurable).
• Checks for pod restarts. If there are none, it assumes that everything is ok and the next iteration happens.

This goes on until only canaries get live traffic. The previous deployment is destroyed and the new one is marked as “production” in the service.

If at any point there are problems with canaries (or restarts), all canary instances are destroyed and all live traffic goes back to the production version.

Pros:

• Version released for a subset of users.
• Convenient for error rate and performance monitoring.
• Fast rollback.

Cons:

• Slow rollout.
• fine tuned traffic distribution can be expensive (99% A/ 1%B = 99 pod A, 1 pod B.

5. Shadow

A shadow deployment consists of releasing version B alongside version A, fork version A’s incoming requests and send them to version B as well without impacting production traffic. This is particularly useful to test production load on a new feature. A rollout of the application is triggered when stability and performance meet the requirements.

This technique is fairly complex to setup and needs special requirements, especially with egress traffic. For example, given a shopping cart platform, if you want to shadow test the payment service you can end-up having customers paying twice for their order. In this case, you can solve it by creating a mocking service that replicates the response from the provider.

Pros:

• Performance testing of the application with production traffic.
• No impact on the user.
• No rollout until the stability and performance of the application meet the requirements.

Cons:

• Expensive as it requires double the resources.
• Not a true user test and can be misleading.
• Complex to set up.
• Requires mocking service for certain case.

6. A/B testing Deployment

A/B testing deployments consist of routing a subset of users to a new functionality under specific conditions. It is usually a technique for making business decisions based on statistics, rather than a deployment strategy. However, it is related and can be implemented by adding extra functionality to a canary deployment so we will briefly discuss it here.

This technique is widely used to test conversion of a given feature and only roll-out the version that converts the most.

Here is a list of conditions that can be used to distribute traffic amongst the versions:

• By browser cookie
• Query parameters
• Geolocalisation
• Technology support: browser version, screen size, operating system, etc.
• Language

Pros:

• Several versions run in parallel.
• Full control over the traffic distribution.

Cons:

• Requires intelligent load balancer.
• Hard to troubleshoot errors for a given session, distributed tracing becomes mandatory.

Authors -

1. Vaibhav Jagtap (A 78)
2. Prem Chavhan (A 36)
3. Rohan Badawe (A 16)
4. Saurav Kendre (A 95)
5. Varad Kulkarni (A 108)
6. Savar Bharadwaj (A 23)