Kubernetes Namespace Quota

-

 


ResourceQuota is an object in Kubernetes that enables administrators to restrict cluster tenants' resource usage per namespace.

Creating a resource quota with HARD cpu, memory,pods and service limit under the namespace "operations".

root@master-node:/kubernetes# kubectl create quota operations-quota --hard=cpu=1,memory=1G,pods=2,services=3

resourcequota/operations-quota created

root@master-node:/kubernetes# kubectl get quota

NAME               AGE   REQUEST                                            LIMIT

operations-quota   5s    cpu: 0/1, memory: 0/1G, pods: 0/2, services: 0/3

root@master-node:/kubernetes#

root@master-node:/kubernetes# kubectl describe quota operations-quota

Name:       operations-quota

Namespace:  operations

Resource    Used  Hard

--------    ----  ----

cpu         0     1

memory      0     1G

pods        0     2

services    0     3

root@master-node:/kubernetes#


root@master-node:/kubernetes# kubectl describe namespace operations

Name:         operations

Labels:       kubernetes.io/metadata.name=operations

Annotations:  <none>

Status:       Active

Resource Quotas

  Name:     operations-quota

  Resource  Used  Hard

  --------  ---   ---

  cpu       0     1

  memory    0     1G

  pods      0     2

  services  0     3

root@master-node:/kubernetes#


Confirm that the namespace has quota assigned to it.

Now, When you try to create a pod (besteffort QOS) with no RAM/CPU mentioned that it will throw error. So, We need to mention RAM/CPU for the pods

root@master-node:/kubernetes# kubectl run pod nginx1 --image=nginx                           

Error from server (Forbidden): pods "pod" is forbidden: failed quota: operations-quota: must specify cpu for: pod; memory for: pod

root@master-node:/kubernetes#

If I delete the quota then I should be able to create pods.

root@master-node:/kubernetes# kubectl delete quota operations-quota

resourcequota "operations-quota" deleted

root@master-node:/kubernetes# kubectl run nginx --image=nginx

pod/nginx created

root@master-node:/kubernetes# kubectl get pods

NAME    READY   STATUS    RESTARTS   AGE

nginx   1/1     Running   0          5s

root@master-node:/kubernetes#

Now I have delete the quota and created 3 more pods under this namespace.

root@master-node:/kubernetes# kubectl get pods

NAME     READY   STATUS    RESTARTS   AGE

nginx    1/1     Running   0          66s

nginx1   1/1     Running   0          10s

nginx2   1/1     Running   0          7s

nginx3   1/1     Running   0          4s

root@master-node:/kubernetes#


Now, this namespace has 4 pods. Lets try to create a quota to this namespace with pods hard limit as 3.


root@master-node:/kubernetes# kubectl create quota operations-quota-alt --hard=pods=3

resourcequota/operations-quota-alt created

root@master-node:/kubernetes# kubectl describe quota operations-quota-alt

Name:       operations-quota-alt

Namespace:  operations

Resource    Used  Hard

--------    ----  ----

pods        4     3

root@master-node:/kubernetes#

root@master-node:/kubernetes# kubectl get pods

NAME     READY   STATUS    RESTARTS   AGE

nginx    1/1     Running   0          2m59s

nginx1   1/1     Running   0          2m3s

nginx2   1/1     Running   0          2m

nginx3   1/1     Running   0          117s

root@master-node:/kubernetes#


Nothing happens to the existing the pods. Let me spin another pod.

root@master-node:/kubernetes# kubectl run nginx4 --image=nginx

Error from server (Forbidden): pods "nginx4" is forbidden: exceeded quota: operations-quota-alt, requested: pods=1, used: pods=4, limited: pods=3

root@master-node:/kubernetes#


Can we attach more than 1 quota to a namespace ?


root@master-node:/kubernetes# kubectl create quota operations-quota-alt1 --hard=pods=5

resourcequota/operations-quota-alt1 created

root@master-node:/kubernetes#


root@master-node:/kubernetes# kubectl describe namespace operations

Name:         operations

Labels:       kubernetes.io/metadata.name=operations

Annotations:  <none>

Status:       Active


Resource Quotas

  Name:     operations-quota-alt

  Resource  Used  Hard

  --------  ---   ---

  pods      4     3

  Name:     operations-quota-alt1

  Resource  Used  Hard

  --------  ---   ---

  pods      4     5


No LimitRange resource.

root@master-node:/kubernetes#


root@master-node:/kubernetes# kubectl run nginx4 --image=nginx

Error from server (Forbidden): pods "nginx4" is forbidden: exceeded quota: operations-quota-alt, requested: pods=1, used: pods=4, limited: pods=3

root@master-node:/kubernetes# kubectl run nginx5 --image=nginx

Error from server (Forbidden): pods "nginx5" is forbidden: exceeded quota: operations-quota-alt, requested: pods=1, used: pods=4, limited: pods=3

root@master-node:/kubernetes#


We can attach more than 1 quota. But, hard limit with lesser value pod count is selected.

But, Lets change the pod limit and try.

operations-quota-alt -> 5 POD.

operations-quota-alt1 -> 3 POD.

root@master-node:/kubernetes# kubectl describe namespace operations

Name:         operations

Labels:       kubernetes.io/metadata.name=operations

Annotations:  <none>

Status:       Active


Resource Quotas

  Name:     operations-quota-alt

  Resource  Used  Hard

  --------  ---   ---

  pods      4     5

  Name:     operations-quota-alt1

  Resource  Used  Hard

  --------  ---   ---

  pods      4     3


No LimitRange resource.

root@master-node:/kubernetes#


root@master-node:/kubernetes# kubectl run nginx4 --image=nginx

Error from server (Forbidden): pods "nginx4" is forbidden: exceeded quota: operations-quota-alt1, requested: pods=1, used: pods=4, limited: pods=3

root@master-node:/kubernetes# kubectl run nginx5 --image=nginx

Error from server (Forbidden): pods "nginx5" is forbidden: exceeded quota: operations-quota-alt1, requested: pods=1, used: pods=4, limited: pods=3

root@master-node:/kubernetes#


Let me change both the pod quota count -> 5 in both the quotas.


root@master-node:/kubernetes# kubectl describe namespace operations

Name:         operations

Labels:       kubernetes.io/metadata.name=operations

Annotations:  <none>

Status:       Active


Resource Quotas

  Name:     operations-quota-alt

  Resource  Used  Hard

  --------  ---   ---

  pods      4     5

  Name:     operations-quota-alt1

  Resource  Used  Hard

  --------  ---   ---

  pods      4     5


No LimitRange resource.

root@master-node:/kubernetes#


root@master-node:/kubernetes# kubectl run nginx4 --image=nginx

pod/nginx4 created

root@master-node:/kubernetes# kubectl run nginx5 --image=nginx

Error from server (Forbidden): pods "nginx5" is forbidden: exceeded quota: operations-quota-alt, requested: pods=1, used: pods=5, limited: pods=5

root@master-node:/kubernetes#


root@master-node:/kubernetes# kubectl get pods

NAME     READY   STATUS    RESTARTS   AGE

nginx    1/1     Running   0          12m

nginx1   1/1     Running   0          11m

nginx2   1/1     Running   0          11m

nginx3   1/1     Running   0          11m

nginx4   1/1     Running   0          16s

root@master-node:/kubernetes#


So, the hard limit always pickups the least count in quotas.


Comments

Post a Comment

Popular posts from this blog

SRE/DevOps Syllabus

-
Image
  DEVOPS/SRE Principles. Git What is Git? Architecture of Git. Working principle of Git. Create and cloning a repo. Version control branching. Version control commit. Version control managing workflows. Git hooks. Git Reflog. Git Stash. Git Cherry Picking. Undoing changes in different states of Git. Git based terraform template management. AWS CI/CD SDLC Automation. Code Commit. Code Build. Code Deployment. Code Pipeline. Elastic Beankstalk. Code Artifact. CodeGuru. Terraform Terraform Basics. Terraform State. Working with Terraform. Terraform with AWS. Remote State. Terraform Provisioners. Terraform Import, Tainting and Debugging. Terraform Modules. Terraform Functions and Conditional Expressions. Automation using Python - AWS AWS lambda. Automating EC2 with Lambda. Automating S3 with Lambda. Automating VPC with Lambda. Cost optimization with Lambda. SNS,SQS and SES with Python. Managing and Automating AWS Security with Python. Kubernetes - CKAD Application Developer. Kubernetes A...
Read more

AWS Code Commit - CI/CD Series Part 1

-
Image
  This is a multi-part series to show how to implement a CI/CD pipeline in AWS. In this post, We will see how to use AWS Code Commit. AWS CodeCommit is a fully managed source control service that allows you to store and manage your code securely in the cloud.  It provides a private Git repository for your code and supports all Git commands and workflows. CodeCommit also integrates with other AWS services like AWS CodePipeline and AWS CodeBuild, allowing you to automate your entire software release process. This can help you streamline your development workflows and improve the overall efficiency of your team. I created a repo from AWS CodeCommit -> Create Repository. And I begin with cloning the repo locally. So that I can make changes and push it. To clone the repo click on "HTTPS" under Clone URL. I make changes to index.html file. Now push it to AWS CodeCommit. Let’s verify the changes in AWS Code commit. Changes are reflected in AWS CodeCommit. We are done with Part ...
Read more

Docker - Preventing IP overlapping

-
Image
  What is IP address overlapping? When the container IP address overlaps with another IP external, then the application running on the container is not accessible from outside due to the IP address overlapping.  NOTE: This IP address overlapping can happen on the bridge interface IP or docker container IP. Let's see how to address this IP overlapping issue on the bridge interface. I already have a running container.  root@demo:~# docker run -d --name web --network mynet1 nginx:alpine 268647e11011d513d2357ce1c81e0f48787038a83d01d028d31a5b29dbdb885b root@demo:~# docker container ls CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS         PORTS     NAMES 268647e11011   nginx:alpine   "/docker-entrypoint.…"   4 seconds ago   Up 3 seconds   80/tcp  ...
Read more