GCP ACE Azure Devops

Today we will have a look at Azure Policy add-on for AkS which was announced GA in MS Ignite 2020.

What is Azure Policy add-on for AKS

Azure Policy add-on for AKS extends Gatekeeper v3, the admission controller webhook for Open Policy Agent (OPA), to apply enforcements and safeguards on the Organisation’s AKS clusters centrally and consistently. Azure Policy provides the capability to manage and report on the compliance state of all Kubernetes clusters from one place. The add-on enables the following functions:

  • Checks with Azure Policy service for policy assignments to the cluster.
  • Deploys policy definitions into the cluster as constraint template and constraint custom resources.
  • Reports auditing and compliance details back to Azure Policy service.

At the time of writing this post, Azure Policy for Kubernetes supports the following cluster environments:

  • Generally Available:
    • Azure Kubernetes Service
  • Preview
    • Azure Arc enabled Kubernetes
    • AKS Engine

Limitations

There are some limitations for using the Policy Add-on, the most notable are :

  • Support for Linux node pools only
  • It supports Only built-in policy definitions

For more info on its current limitations, check this link Limitations

Demo

Pre-requisites:

  • Basic knowledge of Kubernetes commands and components
  • A working AKS cluster (Advanced networking is not needed for this demo)
  • AZ Cli 2.12.0 or higher installed - install Procedure

install the Add-on on an AKS cluster

  • login to the cluster
az login
az aks get-credentials --resource-group $RG_NAME --name $CLUSTER_NAME
az aks enable-addons --addons azure-policy --resource-group $RG_NAME --name $CLUSTER_NAME

Validate that the installation is successful

The azure policy and gatekeeper should be running on the cluster: Run the following commands:

kubectl get pods -n kube-system | grep azure-policy
azure-policy-69cb9f74c7-96cqs           1/1     Running   0          5m22s
azure-policy-webhook-66f9946d9b-qvwsb   1/1     Running   0          5m23s

kubectl get pod -n gatekeeper-system             
NAME                                     READY   STATUS    RESTARTS   AGE
gatekeeper-audit-577989874c-jr97f        1/1     Running   0          3m9s
gatekeeper-controller-5459665bd8-92rlt   1/1     Running   0          3m9s
gatekeeper-controller-5459665bd8-qx55z   1/1     Running   0          3m9s

Ensure that the latest version (v2) of the add-on is installed

az aks show --query addonProfiles.azurepolicy --resource-group $RG_NAME --name $CLUSTER_NAME 
{
  "config": {
    "version": "v2"
  },
  "enabled": true,
  "identity": null
}

Policy Definitions for Kubernetes

To see this built-in Definitions, navigate to the following location in the Azure Portal and filter the Category field to “Kubernetes”

Built-in Policies

Test the policy add-on

We will test the policy using 2 built-in definitions

  • Assign the Definitions to the scope you desire: Subscription, or Resource Group of the AKS cluster
    • Do not allow privileged containers in Kubernetes cluster
    • Enforce internal load balancers in Kubernetes cluster

Test no privileged containers

The following command will try to deploy a pod with the explicit configuration to run as privileged container:

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
  labels:
    app: busybox
spec:
  containers:
  - image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
    securityContext:
      privileged: true
  restartPolicy: Always
EOF

you will receive the following error:

Error from server ([denied by azurepolicy-container-no-privilege-91510313292ddc4bed14] Privileged container is not allowed: busybox, securityContext: {"privileged": true}): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [denied by azurepolicy-container-no-privilege-91510313292ddc4bed14] Privileged container is not allowed: busybox, securityContext: {"privileged": true}

Enforce Internal Load Balancer

Some organisations will want to enforce this policy to prevent Load balancer services from being directly exposed to the Internet

The following command will try to deploy a Kubernetes service of Type Load balancer (Public IP by default on AKS)

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  labels:
    app: nginx-svc
spec:
  type: LoadBalancer
  ports:
  - port: 80
    protocol: TCP
  selector:
    app: nginx
EOF

You will receive the following error:

Error from server ([denied by azurepolicy-load-balancer-no-public-ips-d04d8b03c132e8a572ef] Load Balancers should not have public IPs. azure-load-balancer-internal annotation is required for nginx-svc): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [denied by azurepolicy-load-balancer-no-public-ips-d04d8b03c132e8a572ef] Load Balancers should not have public IPs. azure-load-balancer-internal annotation is required for nginx-svc

Remove the Add-on from the AKS cluster

To remove the add-on from AKS, run the following command:

az aks disable-addons --addons azure-policy -resource-group $RG_NAME --name $CLUSTER_NAME

Conclusion

Azure Policy add-on for AKS is an easy solution to help organisations in their endeavour to increase their AKS Clusters' Security posture by having a tighter control on the conformity of the resources that are deployed within these.

And to make sure that this add-on is deployed on every AKS Cluster that is provisioned, there is a Policy Definition for that purpose as well :)

Name: Deploy Azure Policy Add-on to Azure Kubernetes Service clusters