Application Gateway Ingress Controller for Azure Kubernetes Service

Today we are excited to offer a new solution to bind Azure Kubernetes Service (AKS) and Application Gateway. The new solution provides an open source Application Gateway Ingress Controller (AGIC) for Kubernetes, which makes it possible for AKS customers to leverage Application Gateway to expose their cloud software to the Internet.

Bringing together the benefits of the Azure Kubernetes Service, our managed Kubernetes service, which makes it easy to operate advanced Kubernetes environments and Azure Application Gateway, our native, scalable, and highly available, L7 load balancer has been highly requested by our customers.

How does it work?

Application Gateway Ingress Controller runs in its own pod on the customer’s AKS. Ingress Controller monitors a subset of Kubernetes’ resources for changes. The state of the AKS cluster is translated to Application Gateway specific configuration and applied to the Azure Resource Manager. The continuous re-configuration of Application Gateway ensures uninterrupted flow of traffic to AKS’ services. The diagram below illustrates the flow of state and configuration changes from the Kubernetes API, via Application Gateway Ingress Controller, to Resource Manager and then Application Gateway.

Much like the most popular Kubernetes Ingress Controllers, the Application Gateway Ingress Controller provides several features, leveraging Azure’s native Application Gateway L7 load balancer. To name a few:

◉ URL routing

◉ Cookie-based affinity

◉ Secure Sockets Layer (SSL) termination

◉ End-to-end SSL

◉ Support for public, private, and hybrid web sites

◉ Integrated web application firewall

The architecture of the Application Gateway Ingress Controller differs from that of a traditional in-cluster L7 load balancer. The architectural differences are shown in this diagram:

◉ An in-cluster load balancer performs all data path operations leveraging the Kubernetes cluster’s compute resources. It competes for resources with the business apps it is fronting. In-cluster ingress controllers create Kubernetes Service Resources and leverage kubenet for network traffic. In comparison to Ingress Controller, traffic flows through an extra hop.

◉ Ingress Controller leverages the AKS’ advanced networking, which allocates an IP address for each pod from the subnet shared with Application Gateway. Application Gateway has direct access to all Kubernetes pods. This eliminates the need for data to pass through kubenet. 

Solution performance

As a result of Application Gateway having direct connectivity to the Kubernetes pods, the Application Gateway Ingress Controller can achieve up to 50 percent lower network latency vs in-cluster ingress controllers. Application Gateway is a managed service, backed by Azure virtual machine scale sets. As a result, Application Gateway does not use AKS compute resources for data path processing. It does not share or interfere with the resources allocated to the Kubernetes deployment. Autoscaling Application Gateway at peak times, unlike an in-cluster ingress, will not impede the ability to quickly scale up the apps’ pods. And of course, switching from in-cluster L7 ingress to Application Gateway will immediately decrease the compute load used by AKS.

We compared the performance of an in-cluster ingress controller and Application Gateway Ingress Controller on a three node AKS cluster with a simple web app running 22 pods per node. A total of 66 web app pods shared resources with three in-cluster ingresses – one per node. We configured Application Gateway with an instance count of two. We used Apache Bench to create a total of 100K requests with concurrency set at 3K requests. We launched Apache Bench twice: once pointing it to the SLB fronting the in-cluster ingress controller, and a second time connecting to the public IP of Application Gateway. On this very busy AKS cluster we recorded the mean latency across all requests:

◉ Application Gateway: 480ms per request

◉ In-cluster Ingress: 710ms per request

As proven by the data gathered above, under heavy load, the in-cluster ingress controller has approximately 48 percent higher latency per request compared to Application Gateway ingress. Running the same benchmark on the same cluster but with two web app pods per node, a total of six pods, we observed the in-cluster ingress controller performing with approximately 17 percent higher latency than Application Gateway.