Microsoft AKS updates 2023 - Q2

Within this blog, I want to give an overview of all the feature in Q2 2023 that becomes available in General Availability, Technical Preview or End of Support by Microsoft. This information can be found at Microsoft Azure Updates.

Features that are now supported by Microsoft (GA):

• [General available] Azure Monitor container insights for AKS cluster with ARM64 nodes
  Container insights is a feature designed to monitor the performance of container workloads deployed to the cloud. It gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Azure Monitor container insights is now generally available for AKS clusters with ARM64 nodes. Click here to learn more.
• [General available] Managed identity authentication in Azure Monitor container insights
  Following our public preview announcement last August, we are happy to announce the General Availability of Managed Identity authentication in Container Insights. Managed Identity is a secure and simplified authentication model where our monitoring agent uses the cluster’s managed identity to send data to the Azure Monitor backend. It replaces the existing legacy certificate-based local authentication and removes the requirement of adding a monitoring metrics publisher role to the cluster. Both system-assigned identity and user-assigned identity are supported. Managed Identity will now also be the default authentication mechanism for Container Insights going forward and customers are encouraged to migrate to Managed identity authentication. Click here to learn more.
• [General available] Azure CNI powered by Cilium
  Azure CNI powered by Cilium in now generally available. Azure CNI powered by Cilium is a comprehensive networking solution that combines the control plane of Azure CNI with the dataplane capabilities of Cilium. With faster service routing, enhanced security options, eBPF technology, and compatibility with Overlay mode and Pod subnet mode, it empowers AKS clusters with improved performance, stronger security, and better insights into in-cluster traffic flow. Click here to learn more.
• [General available] Isovalent Cilium Enterprise through Azure Marketplace
  Isovalent Cilium Enterprise is now generally available through Azure Marketplace. With this offering, Isovalent Cilium Enterprise can be deployed on Azure with just a few clicks from the Azure Marketplace. You can either create a new Azure Kubernetes Service (AKS) cluster or seamlessly upgrade an existing AKS cluster running Azure CNI powered by Cilium with Isovalent Cilium Enterprise package. There is zero datapath downtime while upgrading Cilium OSS -> Cilium Enterprise via Azure Marketplace. In addition Azure Marketplace provides unified billing experience while ensuring minimal management overhead for customers in maintaining the upgrades. The tight integration into the Azure platform simplifies operations by enabling auto-upgrades for minor versions. Try it out today on the Azure Marketplace. Explore the tutorial to deploy Isovalent Cilium Enterprise from Azure Marketplace. Click here to learn more about Isovalent Cilium Enterprise Features.
• [General available] AKS DevX Extension Updates for Visual Studio Code
  AKS DevX extension for Visual Studio Code has new updates available. These updates are focused on enhancing your day-to-day life as a developer on Azure Kubernetes Service. The most recent updates bring bug fixes as well as GitHub Action integration. Easily create and deploy GitHub Actions, allowing you to automate your deployments! Click here to learn more.
• [General available] Kubernetes marketplace
  Kubernetes marketplace offer within Azure Marketplace is now generally available. You now have access to a vibrant ecosystem of first-party and third-party solutions that help you build robust enterprise solutions. By building an exclusive marketplace offer type for Kubernetes, we aim to provide a rich catalog of partner and open-source Kubernetes solutions. You can remove adoption barriers through an easy one-click-to-deploy option that installs these solutions to your existing or new AKS cluster. With integrated billing, you can leverage Azure Cost Management to track bills, expenses and predict future expenses. You can procure Kubernetes solutions with speed from trusted procurement channels of Microsoft Cloud Marketplace while accruing towards Microsoft Azure Consumption Commitment (MACC). And you can achieve lifecycle management with upgrades/updates and also receive prompt and timely Enterprise support. The application is vetted and approved for vulnerabilities and exposures to enable you to deploy the applications onto production workloads. Click here to learn more.
• [General available] Generation 2 VM for Windows
  Generation 2 VMs support key features that aren't supported in generation 1 VMs. These features include increased memory, Intel Software Guard Extensions (Intel SGX), and virtualized persistent memory (vPMEM). You can now run Windows workloads on Generation 2 VMs in production to take advantage of these Generation 2 features. Click here to learn more.
• [General available] Azure Linux support in AKS
  Azure Linux as a container host operating system (OS) for Azure Kubernetes Service (AKS) is now generally available. Azure Linux is Microsoft’s Linux distribution of CBL-Mariner supported as a container host OS for AKS. You can now deploy Azure Linux as the node pool host OS in the AKS cluster and build apps on top of the preferred container. Today, Microsoft is extending the AKS container host usage to all AKS customers. Azure Linux as an AKS host OS delivers better performance, can increase the security posture of applications running on AKS clusters and has been optimized to run in Azure. In fact, Azure Linux as a container host OS is optimized for AKS, has a smaller image size thus presenting a smaller attack surface and relies on the same software supply chain used by Microsoft internal engineering teams and services. In addition, AKS on Azure Stack HCI and AKS on Windows Server feature the same container host as well, providing consistency and simplified management across the cloud and the edge. Click here to learn more.
• [General available] Operation Abort in AKS
  AKS now supports aborting a long running operation. This feature, now generally available, allows you to take back control and run another operation seamlessly. This design is supported using the Azure REST API or the Azure CLI. The abort operation supports the following scenarios:
  
  • If a long running operation is stuck or suspected to be in a bad state or failing, the operation can be aborted provided it's the last running operation on the Managed Cluster or agent pool;
    
  • If a long running operation is stuck or failing, that operation can be aborted;
    
  • An operation that was triggered in error can be aborted as long as the operation doesn't reach a terminal state first.
    Click here to learn more.
• [General available] Kubernetes 1.26 support in AKS
  Kubernetes version 1.26 support in AKS is now generally available. You can now take advantage of the v1.26 features in production. To learn more about features in 1.26, see Patch Releases | Kubernetes and Kubernetes 1.26 and Kubernetes 1.26 changelog.
• [General available] Long term support version in AKS
  You can now enable Long Term Support starting with Kubernetes 1.27. Once enabled, this will provide you a 2 year support window for a specific version of Kubernetes. We will provide the ability to have one Kubernetes version enabled for LTS at any one time. For example, the next LTS capable version would be 1.27 plus 2 years (1.33). To learn more, visit: https://aka.ms/aks/upgrades-and-lts.
• [General available] Azure CNI Overlay for Linux
  Azure CNI Overlay simplifies managing cluster nodes and pods within an Azure Virtual Network (VNet) subnet. Nodes are placed directly in the VNet subnet, while pods get IP addresses from a separate private CIDR. An Overlay network handles pod and node traffic within the cluster. To reach external resources, the node's IP address is used for Network Address Translation. This method conserves VNet IP addresses, making it easy to scale your cluster to larger sizes. Plus, the private CIDR can be reused in various AKS clusters, significantly increasing the IP space for containerized AKS applications. To learn more, visit: https://learn.microsoft.com/en-us/azure/aks/azure-cni-overlay.
• [General available] OpenCost for AKS cost visibility
  Born out of Kubecost, Opencost introduces a new community-driven specification and accompanying implementation to bring greater visibility into current and historic Kubernetes spend and resource allocation. OpenCost is an open-source, vendor-neutral CNCF sandbox project that recently became a FinOps Certified Solution. Working in collaboration with project maintainers and community members, AKS has made several contributions to augment OpenCost and enable seamless integration with Azure. To learn more, click on this link.
• [General available] Azure Active Directory workload identity with AKS
  In Azure Kubernetes Service (AKS) today, a preview feature allows you to assign managed identities at the pod-level. This pod-managed identity allows the hosted workload or application access to resources through Azure Active Directory (Azure AD). For example, a workload stores files in Azure Storage, and when it needs to access those files, the pod authenticates itself against the resource as an Azure managed identity. This authentication method is now replaced with Azure Active Directory (Azure AD) workload identities, which integrate with the Kubernetes native capabilities to federate with any external identity providers. This approach is simpler to use and deploy, and overcomes several limitations in Azure AD pod-managed identity:
  
  • Removes the scale and performance issues that existed for identity assignment;
    
  • Supports Kubernetes clusters hosted in any cloud or on-premises;
    
  • Supports both Linux and Windows workloads;
    
  • Removes the need for Custom Resource Definitions and pods that intercept Azure Instance Metadata Service (IMDS) traffic;
    
  • Avoids the complicated and error-prone installation steps such as cluster role assignment from the previous iteration.
    Azure AD workload identity works especially well with the Azure Identity client library using the Azure SDK and the Microsoft Authentication Library(MSAL) if you're using application registration. Your workload can use any of these libraries to seamlessly authenticate and access Azure cloud resources.
    Learn more: https://aka.ms/aks/workloadidentity.
• [General available] Azure CNI Overlay
  Azure CNI overlay in now generally available. Azure CNI overlay addresses performance, scalability and IP exhaustion challenges while using traditional Azure Container Networking Interface (CNI). With Azure CNI overlay AKS clusters can be scaled to very large sizes by assigning pod IP addresses from user defined overlay address space which are logically different from VNet IP address space hosting the cluster nodes. Additionally, user defined private CIDR can be reused in different AKS clusters, truly extending the IP space available for containerized applications in AKS. Pod and node traffic within the cluster use an overlay network via Azure Software Defined Network (SDN) without any additional encapsulation. Network Address Translation (using the node's IP address) is used to reach resources outside the cluster. To learn more, visit: Configure Azure CNI Overlay networking in Azure Kubernetes Service (AKS) - Azure Kubernetes Service | Microsoft Learn.
  

Features that are currently in Public Preview and not yet GA

• [Public Preview] Network Observability add-on on AKS
  Network observability is an important part of maintaining a healthy and performant Kubernetes cluster. By collecting and analyzing data about network traffic, AKS customers can gain insights into how your cluster operate and can identify potential problems before they cause outages or performance degradation. And to enable network observability we have built a network observability add-on plugin that will scrape useful metrics form Kubernetes workloads and emit actionable networking observability data into industry standard Prometheus format, which can then be visualized in Grafana. There are two options available for using Prometheus and Grafana in this context: Azure Managed Prometheus and Grafana or BYO Prometheus and Grafana. Key customer benefits:
  
  • Get access to node-level network metrics like packet drops, connections stats and more;
    
  • Support for all Azure CNIs - AzureCNI and AzureCNI (Powered by Cilium);
    
  • Support for all AKS node types - Linux and Windows;
    
  • Easy deployment using native Azure tools - AKS CLI, ARM templates, PowerShell, etc.;
    
  • Seamless integration with the Azure managed Prometheus and Azure-managed Grafana offerings.
    Read more in the network observability add-on documentation and you can also watch a demo on Microsoft’s Azure YouTube channel.
    
• [Public Preview] Add-on and node image in AKS release tracker
  The AKS release tracker updates are now available in public preview. This release adds specific page for node image regional release progress and release notes. AKS users will get more accurate information about specific node images with the release trackers as the node image updates are decoupled from AKS.. Click here to learn more.
  
• [Public Preview] Automated deployments in AKS now supports Draft
  Public preview of Draft’s integration in automated deployments for AKS is now available. You can now automatically create Dockerfiles and Kubernetes deployment files from source code using automated deployments for Azure Kubernetes Service (AKS) in Azure Portal. Automated deployments simplify the process of setting up an automated workflow for your code releases to your cluster via a GitHub Action. Once connected, every new commit will kick off the workflow, resulting in your application being updated. Click here to learn more.
  
• [Public Preview] Azure Monitor managed service for Prometheus for Azure Arc-enabled Kubernetes
  Azure Monitor managed service for Prometheus is now extending monitoring support for Kubernetes clusters hosted on Azure Arc. Prometheus, the open-source project from the CNCF, is considered the de-facto standard when it comes to monitoring containerized workloads. Running self-managed Prometheus is often a great solution for smaller deployments but scaling this to handle enterprise workloads can be a challenge. Azure Monitor managed service for Prometheus on Azure Arc-enabled Kubernetes allows customers to monitor their Kubernetes clusters running anywhere and keeps the same features as Azure Kubernetes Service (AKS) monitoring. The new fully managed Prometheus compatible service from Azure Monitor delivers the best of what you like about the open-source ecosystem while automating complex tasks such as scaling, high-availability, and long-term data retention. It is available to use as a standalone service from Azure Monitor or as an integrated component of Container Insights and Azure Managed Grafana.
  Click on Azure Monitor managed service for Prometheus Documentation and Collect Prometheus metrics from an Arc-enabled Kubernetes cluster (preview) to learn more.
  
• [Public Preview] Custom node config for AKS Windows Nodepools
  This feature allows you to modify predefined settings on Windows nodepools including imageGcHighThreshold, imageGcLowThreshold, containerLogMaxSizeMB and containerLogMaxFiles. Click here to learn more.
  
• [Public Preview] Azure NetApp Files Standard Network Features - Edit Volumes
  We are announcing the public preview of Edit network features for Azure NetApp Files volumes in select regions which has been made possible by innovative hardware and software integration. Standard Network Features provide you with an enhanced Virtual Networking experience for a seamless and consistent experience along with security posture for Azure NetApp Files. You are now able to edit existing ANF volumes and upgrading Basic network features to Standard network features. Upon choosing Standard network features, you can now take advantage of the below supported following new features for ANF volumes/delegated subnets:
  
  • Increased IP limits for the Vnets with ANF volumes at par with VMs to enable customers to provision ANF volumes in their existing topologies/architectures. This eliminates the need for you to rearchitect you network topologies to use ANF for workloads like VDI, WVD or AKS;
    
  • Enhanced network security with support for Network Security Groups on the Azure NetApp Files delegated subnet. NSGs on the ANF delegated subnets have been a long standing ask from customers to meet enterprise security requirements;
    
  • Enhanced network control with support for User-defined routes to and from Azure NetApp Files delegated subnets. You can now direct the traffic to and from Azure NetApp Files via your choice of Network Virtual Appliances for traffic inspection;
    
  • Connectivity over Active/Active VPN gateway setup for highly available connectivity to ANF from on-prem;
    
  • ExpressRoute FastPath connectivity to Azure NetApp Files. FastPath is designed to improve the data path performance (low latency and high bandwidth connectivity) between on-premises network and Azure virtual network.
    Click on Configure network features for an Azure NetApp Files volume | Microsoft Learn and Guidelines for Azure NetApp Files network planning | Microsoft Learn to learn more.
    
• [Public Preview] Azure Container Storage
  Azure Container Storage, now in preview, is a unique volume management service built natively for containers. It provides a consistent experience across different types of storage offerings, including Managed option (backed by Azure Elastic SAN), Azure Disks, and ephemeral disk on container services. This simplifies the deployment of persistent volumes and offers a highly scalable, cost-effective, high-performance and resilient storage solution. With Azure Container Storage, you can easily create and manage block storage volumes for production-scale stateful container applications and run them on Kubernetes, ensuring consistent experiences across different environments. The solution is optimized to enhance the performance of stateful workloads on Azure Kubernetes Service (AKS) clusters by accelerating the deployment of stateful containers with persistent volumes and improving quality with reduced pod failover time through fast attach/detach. Additionally, by efficiently deploying and managing persistent volumes on backend storage options, you can reduce the total cost of ownership (TCO) associated with container storage. Learn more by reading the blog and refer to the documentation for additional details, including how you can request access to the preview.
• [Public Preview] AKS service mesh addon for Istio
  Azure Kubernetes Service (AKS) addon for service mesh based on Istio is now available in public preview. Istio addresses the challenges developers and operators face with a distributed or microservices architecture and can be used to streamline traffic management, security, and observability for service-to-service communication scenarios. The AKS addon for service mesh builds on top of open source Istio and provides additional benefits such as compatibility testing done between Istio with supported versions of AKS, managed external/internal ingresses, and scaling of Istio control plane components. To learn more about this addon and get started, visit: https://aka.ms/asm-aks-addon-docs. To learn more about the roadmap for service mesh area, visit: https://aka.ms/asm-roadmap.
• [Public Preview] Fail Fast Upgrade on API Breaking change detection
  Azure Kubernetes Service (AKS) now supports fail fast on minor version change cluster upgrades. This feature alerts you with an error message if it detects usage on deprecated APIs in the intended goal version provided you are using the latest preview API version. Detecting the change at this stage saves you from having to spend time on post upgrade workload troubleshooting. The deprecated APIs are as per the Deprecated API Migration Guide | Kubernetes.
• [Public Preview] Isovalent Cilium Enterprise through Azure Marketplace
  Isovalent Cilium Enterprise platform is now available through Azure Marketplace. With this offering, Isovalent Cilium Enterprise can be deployed on Azure with just a few clicks from the Azure Marketplace. You can either create a new Azure Kubernetes Service (AKS) cluster orseamless upgrade an existing AKS cluster running Azure CNI powered by Cilium with Isovalent Cilium Enterprise package. There is zero datapath downtime while upgrading Cilium OSS -> Cilium Enterprise via Azure Marketplace. In addition Azure Marketplace provides unified billing experience and an integrated experience for your Isovalent Cilium Enterprise usage while ensuring minimal management overhead for customers in maintaining the upgrades. The tight integration into the Azure platform simplifies operations by enabling auto-upgrades for minor versions. Try it out today on the Azure Marketplace. To learn more on Isovalent Cilium Enterprise Features, read Isovalent Blog Post and Isovalent Cilium Enterrpise Product Page.
• [Public Preview] Node Resource Group (NRG) lockdown
  Node Resource Group Lockdown removes the ability for customers to modify resources created as part of the AKS cluster. Currently customers can directly modify and delete resources created by AKS, which can lead to an unstable environment. To reduce these scenarios, NRG Lockdown applies a deny assignment to the node resource group, and any changes must happen through the AKS control plane. Click here to learn more.
  

For more information about the features that are coming out, please refer to the public roadmap of Microsoft AKS team.