In this post we will go through the deployment of the Application Gateway Ingress Controller (AGIC) into AKS and the solution it provides.
This is an attempt to simplify its deployment by providing scripts and resources which are available on my Github Repo Adeelku/aks-agic and will be demonstrated in this post.
It allows the use of a single Application Gateway Ingress Controller to control multiple AKS clusters. It also helps eliminate the need to have another load balancer/public IP in front of AKS cluster and avoids multiple hops before requests reach the AKS cluster. Application Gateway talks to pods directly using their private IP and does not require NodePort or KubeProxy services. This also increases the deployment’s performance.
Ingress Controller is exclusively supported by Standard_v2 and WAF_v2 SKUs, which also brings autoscaling benefits. Application Gateway can react in response to an increase or decrease in traffic load and scale accordingly, without consuming any resources from the AKS cluster.
Using Application Gateway helps protect the AKS cluster by providing TLS policy and Web Application Firewall (WAF) functionality.
What is an Ingress Controller?
An Ingress is a Kubernetes API object that manages external access to the services in a cluster, typically HTTP. It can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
An ingress controller is a piece of software that provides reverse proxy, configurable traffic routing, and TLS termination for Kubernetes services. Kubernetes ingress resources are used to configure the ingress rules and routes for individual Kubernetes services. Using an ingress controller and ingress rules, a single IP address can be used to route traffic to multiple services in a Kubernetes cluster.
Popular ingress controllers are NGINX, Traefik, HAProxy, Envoy.
Ingress Controllers for AKS
it is possible to configure Ingress Cootrollers for AKS via different solutions:
HTTP application routing
The HTTP application routing add-on simplifie access to applications deployed to Azure Kubernetes Service (AKS) cluster. When the solution’s enabled, it configures an Ingress controller in the AKS cluster. As applications are deployed, the solution also creates publicly accessible DNS names for application endpoints.
⚠️ The HTTP application routing add-on is designed to enable Ops quickly create an ingress controller and access the applications. This add-on is not recommended for production use.
Production-ready Ingress controllers
This is the native Kubernetes solution to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource
The Application Gateway Ingress Controller (AGIC) enables exposing applications running within AKS to the Internet by leveraging Azure’s native Application Gateway L7 load-balancer. AGIC monitors the Kubernetes cluster on which it is hosted and continuously updates an Application Gateway so that selected services are exposed to the Internet.
AGIC is configured via the Kubernetes Ingress resource, along with Service and Deployments/Pods.
- yq installed on your developement machine - used for yaml templating
sudo add-apt-repository ppa:rmescandon/yq sudo apt install yq
- Basic knowledge of Kubernetes commands and components
- A working AKS cluster with advanced networking enabled.
- You can use the ./aks/create_aks_adv_net.sh AZ CLI script from my Github repo adeelku/aks-modules
- An Azure Application Gateway v2 with public IP in the same vnet where the AKS Cluster lives
- You can use the ./application_gtw/create_aag.sh AZ CLI script from my Github repo adeelku/aks-modules
- Helm v3 deployed onto your developement machine - install Procedure
install Helm on Linux (amd64)
VERSION="v3.1.1" wget https://get.helm.sh/helm-$VERSION-linux-amd64.tar.gz tar -zxvf helm-$VERSION-linux-amd64.tar.gz sudo mv linux-amd64/helm /usr/local/bin/helm rm -rf linux-amd64/ rm helm-$VERSION-linux-amd64.tar.gz
Now we will deploy and configure the different components needed to set-up Azure Application Ingress Controller:
- Create Azure Managed Identity (needs to be deployed into the MC_ Resource group automatically created when creating AKS Cluster)
- This Managed Identity will be used by the AAD Pod Idenity
- Grant this Managed Indentity:
- Contributor eole on the Application Gateway
- Reader role on the Application Gateway Resource Group
- Deploy the Kubernetes components:
- Deploy the Managed Idenity Controller and Node Managed Identity
- Deploy application gateway ingress controller using Helm
To simplify automation of the deployment and configuration process by retrieving the Azure Resoucres infornation and using them dynamically, I prepared a Git Repo for this purpose.
- Clone the repo:
git clone https://github.com/Adeelku/aks-agic.git . ├── k8s_yaml │ ├── aadpodidentitybinding.yaml │ ├── aadpodidentity.yaml │ └── sample_app.yaml ├── LICENSE ├── README.md └── scripts ├── deploy_into_k8s.sh └── params.sh
- Update the file ./scripts/params.sh to match your AKS and Application Gateway configuration
export RG_NAME="aks-poc-rg" # AKS Resource Group export CLUSTER_NAME="aks-poc" # AKS Cluster Name export MSI_NAME="aks-poc-msi" # Managed Identity Desired Name export LOCATION="canadacentral" # Location of AKS Cluster export NET_RG_NAME="net-poc-rg" # Resource Group of Application Gateway export AAG_NAME="aag-poc" # Application Gateway Name
- Deploy using the script ./scripts/deploy_into_k8s.sh
a. Managed identity will be created b. Template files for the Managed Idenity Controller and Node Managed Identity will be populated with the information needed
apiVersion: aadpodidentity.k8s.io/v1 kind: AzureIdentity metadata: name: spec: type: 0 ResourceID: ClientID:
apiVersion: aadpodidentity.k8s.io/v1 kind: AzureIdentityBinding metadata: name: poc-identity-binding spec: AzureIdentity: Selector: poc-msi
c. Azure Identity Installed and bind in Kubernetes d. Application gateway ingress controller deployed using helm and the Azure resources information
helm upgrade -i ingress-agic application-gateway-kubernetes-ingress/ingress-azure \ --namespace default \ --debug \ --set appgw.name=$AAG_NAME \ --set appgw.resourceGroup=$NET_RG_NAME \ --set appgw.subscriptionId=$AAG_SUBID \ --set appgw.shared=false \ --set appgw.usePrivateIP=false \ --set armAuth.type=aadPodIdentity \ --set armAuth.identityResourceID=$MSI_R_ID \ --set armAuth.identityClientID=$MSI_C_ID \ --set rbac.enabled=true \ --set verbosityLevel=3 \ --set kubernetes.watchNamespace=default \ --set aksClusterConfiguration.apiServerAddress=$AKS_API
e. Finally, Resetting template files and removing microsoft managed identity info
When the deployment is successfull, all the Kubernetes components will be in a running state:
Create a sample application to validate that agic is woring properly
kubectl apply -f k8s_yaml/sample_app.yaml pod/aspnetapp created service/aspnetapp created ingress.extensions/aspnetapp created
Get the Public IP Address of your Aplication Gatway and navigate to the Application
az network public-ip show -n $pub-aag -g $net-poc-rg --query ipAddress -o tsv
And that’s it! you have got agic deployed into AKS.
⚠️ Of Course in Production you want to be using Application Gateway with Firewall enabled (SKU = WAF_v2) and secure traffic using SSL/TLS certificates