Kubernetes HTTP Request Flow
As we learned in earlier chapters, when we run any kubectl command, the command is translated into an HTTP API request in JSON format and is sent to the API server. Then, the API server returns a response to the client, along with any requested information. The following diagram shows the API request life cycle and what happens inside the API server when it receives a request:
Figure 4.2: API server HTTP request flow
As you can see in the preceding figure, the HTTP request goes through the authentication, authorization, and admission control stages. We will take a look at each of these in the following subtopics.
Authentication
In Kubernetes, every API call needs to authenticate with the API server, regardless of whether it comes from outside the cluster, such as those made by kubectl, or a process inside the cluster, such as those made by kubelet.
When an HTTP request is sent to the API server, the API server needs to authenticate the client sending this request. The HTTP request will contain the information required for authentication, such as the username, user ID, and group. The authentication method will be determined by either the header or the certificate of the request. To deal with these different methods, the API server has different authentication plugins, such as ServiceAccount tokens, which are used to authenticate ServiceAccounts, and at least one other method to authenticate users, such as X.509 client certificates.
Note
The cluster administrator usually defines authentication plugins during cluster creation. You can learn more about the various authentication strategies and authentication plugins at https://kubernetes.io/docs/reference/access-authn-authz/authentication/.
We will take a look at the implementation of certificate-based authentication in Chapter 11, Build Your Own HA Cluster.
The API server will call those plugins one by one until one of them authenticates the request. If all of them fail, then the authentication fails. If the authentication succeeds, then the authentication phase is complete and the request proceeds to the authorization phase.
Authorization
After authentication is successful, the attributes from the HTTP request are sent to the authorization plugin to determine whether the user is permitted to perform the requested action. There are various levels of privileges that different users may have; for example, can a given user create a pod in the requested namespace? Can the user delete a Deployment? These kinds of decisions are made in the authorization phase.
Consider an example where you have two users. A user called ReadOnlyUser (just a hypothetical name) should be allowed to list pods in the default namespace only, and ClusterAdmin (another hypothetical name) should be able to perform all tasks across all namespaces:
Figure 4.3: Privileges for our two users
To understand this better, take a look at the following demonstration:
Note
We will not pe into too much detail about how to create users as this will be discussed in Chapter 13, Runtime and Network Security in Kubernetes. For this demonstration, the users, along with their permissions, are already set up, and the limitation of their privileges is demonstrated by switching contexts.
Figure 4.4: Demonstrating different user privileges
Notice, from the preceding screenshot, that the ReadOnlyUser can only list pods in the default namespace, but when trying to perform other tasks, such as deleting a pod in the default namespace or listing pods in other namespaces, the user will get a Forbidden error. This Forbidden error is returned by the authorization plugin.
kubectl provides a tool that you can call by using kubectl auth can-i to check whether an action is allowed for the current user.
Let's consider the following examples in the context of the previous demonstration. Let's say that the ReadOnlyUser runs the following commands:
kubectl auth can-i get pods --all-namespaces
kubectl auth can-i get pods -n default
The user should see the following responses:
Figure 4.5: Checking privileges for ReadOnlyUser
Now, after switching context, let's say that the ClusterAdmin user runs the following commands:
kubectl auth can-i delete pods
kubectl auth can-i get pods
kubectl auth can-i get pods --all-namespaces
The user should see the following response:
Figure 4.6: Checking privileges for ClusterAdmin
Unlike authentication phase modules, authorization modules are checked in sequence. If multiple authorization modules are configured, and if any authorizer approves or denies a request, that decision is immediately returned, and no other authorizer will be contacted.
Admission Control
After the request is authenticated and authorized, it goes to the admission control modules. These modules can modify or reject requests. If the request is only trying to perform a READ operation, it bypasses this stage; but if it is trying to create, modify, or delete, it will be sent to the admission controller plugins. Kubernetes comes with a set of predefined admission controllers, although you can define custom admission controllers as well.
These plugins may modify the incoming object, in some cases to apply system-configured defaults or even to deny the request. Like authorization modules, if any admission controller module rejects the request, then the request is dropped and it will not process further.
Some examples are as follows:
- If we configure a custom rule that every object should have a label (which you will learn how to do in Chapter 16, Kubernetes Admission Controllers), then any request to create an object without a label will be rejected by the admission controllers.
- When you delete a namespace, it goes to the Terminating state, where Kubernetes will try to evict all the resources in it before deleting it. So, we cannot create any new objects in this namespace. NamespaceLifecycle is what prevents that.
- When a client tries to create a resource in a namespace that does not exist, the NamespaceExists admission controller rejects the request.
Out of the different modules included in Kubernetes, not all of the admission control modules are enabled by default, and the default modules usually change based on the Kubernetes version. Providers of cloud-based Kubernetes solutions, such as Amazon Web Services (AWS), Google, and Azure, control which plugins can be enabled by default. Cluster administrators can also decide which modules to enable or disable when initializing the API server. By using the --enable-admission-plugins flag, administrators can control which modules should be enabled other than the default ones. On the other hand, the --disable-admission-plugins flag controls which modules from the default modules should be disabled.
Note
You will learn more about admission controllers, including creating custom ones, in Chapter 16, Kubernetes Admission Controllers.
As you will recall from Chapter 2, An Overview of Kubernetes, when we created a cluster using the minikube start command, Minikube enabled several modules for us by default. Let's take a closer look at that in the next exercise in which we will not only view the different API modules enabled for us by default but also start Minikube with a custom set of modules.
Exercise 4.01: Starting Minikube with a Custom Set of Modules
In this exercise, we will take a look at how to view the different API modules enabled for our instance of Minikube, and then restart Minikube using a custom set of API modules:
- If Minikube is not already running on your machine, start it up by using the following command:
minikube start
You should see the following response:
Figure 4.7: Starting up Minikube
- Now, let's see which modules are enabled by default. Use the following command:
kubectl describe pod kube-apiserver-minikube -n kube-system | grep enable-admission-plugins
You should see the following response:
Figure 4.8: Default modules enabled in Minikube
As you can observe from the preceding output, Minikube has enabled the following modules for us: NamespaceLifecycle, LimitRanger, ServiceAccount, DefaultStorageClass, DefaultTolerationSeconds, NodeRestriction, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, and ResourceQuota.
Note
To know more about modules, please refer the following link: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
- Another way to check the modules is to view the API server manifest by running the following command:
kubectl exec -it kube-apiserver-minikube -n kube-system -- kube-apiserver -h | grep "enable-admission-plugins" | grep -vi deprecated
Note
We used grep -vi deprecated because there is another flag, --admission-control, that we are discarding from the output, as this flag will be deprecated in future versions.
kubectl has the exec command, which allows us to execute a command to our running pods. This command will execute kube-apiserver -h inside our kube-apiserver-minikube pod and return the output to our shell:
Figure 4.9: Checking the modules enabled by default in Minikube
- Now, we will start Minikube with our desired configuration. Use the following command:
minikube start --extra-config=apiserver.enable-admission-plugins="LimitRanger,NamespaceExists,NamespaceLifecycle,ResourceQuota,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook"
As you can see here, the minikube start command has the --extra-config configurator flag, which allows us to pass additional configurations to our cluster installation. In our case, we can use the --extra-config flag, along with --enable-admission-plugins, and specify the plugins we need to enable. Our command should produce this output:
Figure 4.10: Restarting Minikube with a custom set of modules
- Now, let's compare this instance of Minikube with our earlier one. Use the following command:
kubectl describe pod kube-apiserver-minikube -n kube-system | grep enable-admission-plugins
You should see the following response:
Figure 4.11: Checking a custom set of modules for Minikube
If you compare the set of modules seen here to the ones in Figure 4.7, you will notice that only the specified plugins were enabled; while the DefaultTolerationSeconds, NodeRestriction, and ValidatingAdmissionWebhook modules are no longer enabled.
Note
You can revert to the default configurations in Minikube by running minikube start again.
Validation
After letting the request pass through all three stages, the API server then validates the object—that is, it checks whether the object specification, which is carried in JSON format in the response body, meets the required format and standard.
After successful validation, the API server stores the object in the etcd datastore and returns a response to the client. After that, as you learned in Chapter 2, An Overview of Kubernetes, other components, such as the scheduler and the controller manager, take over to find a suitable node and actually implement the object on your cluster.