Microsoft AKS updates 2023 - Q4

Dec 31, 2023 · 14 min read · Kubernetes AKS ·

Share on:

Within this blog, I want to give an overview of all the feature in Q4 2023 that becomes available in General Availability, Technical Preview or End of Support by Microsoft. This information can be found at Microsoft Azure Updates.

Features that are now supported by Microsoft (GA):

[General available] AKS support for API breaking change detection
Azure Kubernetes Service (AKS) now supports fail fast on minor version change of Kubernetes cluster upgrades. This feature alerts you with an error message if it detects usage of deprecated Kubernetes standard APIs in the intended goal version provided you are using the latest API version. Detecting the change at this stage saves you from having to spend time on post upgrade workload troubleshooting. The deprecated APIs are as per the Kubernetes Deprecated API Migration Guide: https://kubernetes.io/docs/reference/using-api/deprecation-guide/. Click here to learn more.
[General available] Azure Backup for AKS
We are excited to announce the Azure Backup for AKS is now Generally Available. With our Azure native, Kubernetes aware, Enterprise ready backup for containerized applications deployed on Azure Kubernetes Service (AKS), customers can now simply and safely protect their mission critical workloads. Azure Backup for AKS helps IT admins with its application-centric automated backups and simple restores for AKS clusters, all within single pane of glass to monitor, govern and manage. With this release we brought in support for:
1. For applications running databases like MySQL and MongoDB in AKS cluster, application consistent snapshots are required for backup. AKS Backup supports this via custom hooks;
2. You can now restore your backups to AKS clusters running in a different subscription. This helps in key scenarios like application migration;
3. AKS Backup supports BYO model and now you can backup Azure Disks statically provisioned and also clusters using an User Identity for RBAC operations.
  
  Azure Backup for AKS helps organizations protect their stateful applications and data, mitigating risks of data loss or system downtime. The solution is simple, easy to use, and provides peace of mind for IT Managers responsible for managing stateful applications running on AKS clusters. To get started with Azure Backup for AKS, visit the Azure documentation, and start protecting your AKS clusters.
[General available] Collect Syslog from AKS nodes using Azure Monitor Container Insights
The ability to collect Syslog from Linux-based host nodes in AKS is now generally available. The GA release comes with reliability improvements, an out-of-box dashboard in Azure Managed Grafana, and the ability to send Syslog data to Microsoft Sentinel. Syslog is a popular message logging standard that can be used across a variety of devices like servers, Virtual Machines, routers, and other devices. Enterprises commonly use syslog for collecting logs in on-premise, and IaaS workloads. Customers can now collect Syslog from their AKS Clusters using Azure Monitor - Container insights. Combined with SIEM systems (Microsoft Sentinel) and observability tools (Azure Monitor), syslog collection enables security monitoring and troubleshooting for AKS clusters. Read more about these updates in our blog post. See Syslog collection with Container Insights to learn more. IMPORTANT NOTE: Due to slower rollouts towards the year end, the agent version with the GA changes will not be in all regions until January 2024. Agent versions 3.1.16 and above have Syslog GA changes. Please check the agent version before enabling in production.
[General available] Kubernetes AI toolchain operator
You can now run specialized machine learning workloads like large language models (LLMs) on Azure Kubernetes Service (AKS) more cost-effectively and with less manual configuration. The initial release of Kubernetes AI toolchain operator, an open source project, automates LLM model deployment on AKS across available CPU and GPU resources by selecting optimally sized infrastructure for the model. It makes it possible to easily split inferencing across multiple lower-GPU count VMs, increasing the number of Azure regions where workloads can run, eliminating wait times for higher GPU-count VMs, and lowering overall cost. You can also choose from preset models with images hosted by AKS, significantly reducing overall inference service setup time. Click here to learn more.
[General available] Azure Kubernetes Fleet Manager
Azure Kubernetes Fleet Manager (Fleet) is now generally available. It enables multi-cluster and at-scale scenarios for Azure Kubernetes Service (AKS) clusters. Platform admins who are managing Kubernetes fleets with a large number of clusters often face challenges staging their updates across clusters in a safe and predictable way. Azure Kubernetes Fleet Manager allows admins to orchestrate updates across multiple clusters by using update runs, stages, and groups. Azure Kubernetes Fleet Manager now also offers an update orchestration feature. As part of update orchestration, you are now able to store templates for update runs in the form of update strategies and the ability to set node image consistency as a desired outcome throughout the update run. Click here to learn more.
[General available] Provider for running Karpenter on Azure Kubernetes Service (AKS)
Karpenter is an open-source node provisioning project for Kubernetes. A new provider for running Karpenter on Azure Kubernetes Service is now available as an open-source project. Karpenter improves the efficiency and cost of running workloads on Kubernetes clusters by:
- Watching for pods that the Kubernetes scheduler has marked as unschedulable;
- Evaluating scheduling constraints (resource requests, nodeselectors, affinities, tolerations, and topology spread constraints) requested by the pods;
- Provisioning nodes that meet the requirements of the pods;
- Scheduling the pods to run on the new nodes;
- Removing the nodes when the nodes are no longer needed.
  Click here to learn more.
[General available] Kubernetes Event-driven Autoscaling (KEDA) Add-on for AKS
The Kubernetes Event-driven Autoscaling(KEDA) add-on or AKS is now generally available. KEDA is a single-purpose and lightweight component that strives to make application autoscaling simple and is a CNCF graduated project. It applies event-driven autoscaling to scale your application to meet demand in a sustainable and cost-efficient manner with scale-to-zero. The KEDA add-on makes it even easier by deploying a managed KEDA installation, and provides a rich catalog of more than 60 KEDA scalers that can be used to scale applications on AKS cluster. Click here to learn more.
[General available] Kubernetes 1.28 support in Azure Kubernetes Service (AKS)
AKS support for Kubernetes version 1.28 is now generally available. Kubernetes 1.28 contains over 40 features and enhancements with continued improvement on reliability and performance. To read more about Kubernetes 1.28, check out the open-source blog post and the changelog.
[General available] Application routing add-on for Azure Kubernetes Service (AKS)
Application routing add-on for Azure Kubernetes Service (AKS) is now generally available. App routing is the easiest way to get your web application up and running in AKS securely while removing the complexity of setting up an ingress controller, certificate, and DNS management all while constructing a solid foundation that enterprises can utilize as their demands grow. App routing offers a supported and managed ingress controller powered by the ingress-nginx project. Click here to learn more.
[General available] Kube-reserved resource optimization in Azure Kubernetes Service (AKS)
Reserved space contains resources set aside on a node, such as system daemons that back Kubernetes and the operating system itself. If these resources are not allocated sufficient reserved space, pods and system daemons will compete. This competition leads to starvation on the node. Azure Kubernetes Service (AKS) addresses this issue by enforcing a rate at which it reserves space, as detailed by the Kube-reserved flag. This flag shows the resource reservation for Kubernetes system daemons including kubelet, container runtime, and more. Optimized AKS reservation logic reduces Kube-reserved memory by up to 20% depending on the node configuration and will apply to everyone. Click here to learn more.
[General available] Bring our own keys (BYOK) on Ephemeral OS Disk
AKS nodes support for using custom managed keys for encryption of both the OS and data disks of clusters is now generally available. Azure storage encrypts all data in a storage account at rest. By default, data is encrypted with Microsoft-managed keys. For more control over encryption keys, you can supply your own managed keys to use for encryption at rest for both the OS and data disks for your AKS clusters. Click here to learn more.

Features that are currently in Public Preview and not yet GA

[Public Preview] New AKS cost views for standard and premium tier clusters
You can now get granular visibility into Kubernetes costs in Cost analysis within Azure portal. You can view the aggregated costs for all your clusters in a subscription and costs of all the namespaces for your clusters. All you need to do is to enable the AKS cost analysis add-on for your clusters. The AKS cost analysis add-on is built on top of OpenCost, an open-source Cloud Native Computing Foundation Sandbox project for usage data collection, which gets reconciled with your Azure billing data. Please refer to the articles below for installation of the add-on and accessing the views. To learn more, click here or here.
[Public Preview] Node autoprovision support in AKS
Azure Kubernetes Service (AKS) now supports Node autoprovision (NAP) in public preview. This feature will provision the right VMs for your workloads based on the resources needed to efficiently allocate infrastructure. This greatly reduces the burden on you to design your node pool configuration ahead of workloads being deployed. NAP also comes with consolidation, which efficiently reschedules your workloads on the right size of virtual machines greatly reducing running costs for your applications. To learn more, click here.
[Public Preview] Azure Container Storage is now available with Azure Linux container host
Azure Container Storage in preview, provides highly scalable, cost-effective volumes for production-scale stateful container applications seamlessly via Azure Kubernetes Service (AKS) CLI. You can now leverage Azure Container Storage with AKS clusters or nodepools using Azure Linux container host. To achieve this, you can simply create a new managed Kubernetes cluster by selecting the OS SKU as Azure Linux and enabling Azure Container Storage, directly as part of the AKS cluster create or update CLI experience. To learn more, read the documentation, blog, and watch the video.
[Public Preview] Cost analysis add-on for AKS
AKS Cost Analysis add-on for AKS, now available in public preview, is a native Azure Portal experience that breaks down underlying cluster infrastructure costs by Kubernetes specific constructs such as cluster and namespace. The add-on is built on top of OpenCost, a CNCF Sandbox project, and is available for Standard and Premium tier AKS clusters. You can access cost allocation data directly in Azure Portal UI. With deeper visibility, you are better equipped to tackle everyday cost monitoring, allocation, and cost optimization scenarios. Click here to learn more.
[Public Preview] Confidential containers on Azure Kubernetes Service (AKS)
Confidential containers on Azure Kubernetes Service (AKS), leveraging the Kata confidential containers open-source project, is now in public preview. It enables you to run individual pods in their own trusted execution environment (TEE) with hardware-based confidentiality and integrity protections for your container workloads while in use in memory. Confidential containers on AKS is supported as a new SKU that you can select when deploying your workload and will provide you with the following benefits for workloads processing highly sensitive data:
- Ability to lift and shift workloads to a confidential environment without needing to take any dependencies on any confidential computing libraries;
- In-memory encryption of data with a hardware based dedicated key per container group helping to guard against attacks from malicious OS or hypervisor components, and even your own tenant administrators;
- Support for remote attestation to enable a relying party to verify that a service is running in a TEE before processing any sensitive data. As part of confidential containers on AKS, an agent will validate the authenticity of the hardware and application components which can be verified through a remote attestation service before any sensitive data is released to the TEE.
  Click here to learn more.
[Public Preview] Dual-stack networking in Azure CNI Overlay for AKS
Dual-stack support in Azure CNI Overlay for AKS is now in public preview. This preview release enhances AKS networking capabilities, allowing both IPv4 and IPv6 addresses to coexist and operate within the same Cluster, offering greater flexibility and connectivity options. The Azure CNI Overlay model, known for its performance and scaling merits, now extends its advantages to dual-stack networking, ensuring seamless communication with external systems operating on either IP address family. This feature not only addresses the growing demand for IPv6 support but also facilitates a smooth transition for enterprises adapting to the evolving network standards. Click here to learn more.
[Public Preview] Enhancements for Istio-based service mesh add-on for AKS
Istio-based service mesh add-on for AKS, currently in public preview, offers new features related to security and upgrades. Click here to learn more.
[Public Preview] Artifact streaming support in Azure Kubernetes Service (AKS)
You can now accelerate your containerized workloads on Azure Kubernetes Service (AKS) using Azure Container Registry (ACR) artifact streaming. Artifact streaming will allow you to scale your workloads without having to fully wait for images to be pulled into the clusters. Artifact Streaming is available now for Linux-based container images via the ACR and AKS Preview API. CLI and Portal support will be available in the next two weeks. Click here to learn more.
[Public Preview] Image integrity support in Azure Kubernetes Service (AKS)
Using signed container images helps verify that your deployments are built from a trusted entity and that images haven't been tampered with since their creation. Image integrity for AKS allows you to add an Azure Policy built-in definition to verify that only signed images are deployed to your AKS clusters. Click here to learn more.
[Public Preview] Azure Container Storage in AKS
Azure Container Storage (in preview) provides highly scalable, cost-effective persistent volumes, built natively for containers. You can now deploy and use Azure Container Storage seamlessly via the Azure Kubernetes Service (AKS) cluster create or update preview experience in 26 regions, simplifying volume provisioning and management of stateful container applications on AKS. Azure Container Storage provides rapid scale out of volumes, reduced pod failover time, reduced total cost of ownership, consistently across different block storage options, including ephemeral disks, Azure Disks and Azure Elastic SAN. With this preview update you can now:
- Configure multi-zone storage pools for high availability and redundancy;
- Secure storage pools using server-side encryption with customer managed keys;
- Dynamically resize volumes;
- Protect and recover volumes in a storage pool using snapshot and clone.
  To learn more, read the documentation, blog, and watch the video.
[Public Preview] Azure integration with Canonical’s Snapshot Service for safe deployment
Microsoft and Canonical have partnered to make it easier for you to stay current with Linux operating system (OS) updates and increase the security and resiliency of Canonical workloads on Azure. Azure is the first cloud provider to collaborate with Canonical to integrate its snapshot service. Azure Guest Patching Service (AzGPS) and Azure Kubernetes Service (AKS) will leverage the new capability to apply the same update consistently on your fleet across regions via safe deployment principles (SDP). Click beneath to learn more:
[Public Preview] Disable Secure Shell(SSH) support in AKS
Secure Shell (SSH) is currently on by default for AKS provisioned nodes, and you must disable SSH manually. This public preview feature allows you to disable or enable SSH. This gives you the ability to secure your cluster and reduce the attack surface. Click here to learn more.
[Public Preview] Regional Disaster Recovery by Azure Backup for AKS
Azure Backup for AKS enables customers to protect their containerized workloads along with application data deployed on AKS clusters. The solution allows you to configure scheduled backups of your AKS clusters and restore them in same or alternate cluster in the scenarios like Operational Recovery, Accidental Deletion and Application Migration. Customers are also looking to utilize their AKS backups to recover application during a regional disaster recovery and also follow industry-wide best practice of 3-2-1 backup strategy. With this intent, Azure Backup service is announcing private preview of AKS Backup - Regional Disaster Recovery Capability. Using this feature you can:
- Recover AKS cluster from your backups in a secondary region as an Azure Paired Region in case of a regional disaster;
- Store Backup Copy offsite i.e. a Vault Store as per 3-2-1 backup strategy and have ability to restore in case your tenant gets compromised;
- Retain data for a long duration for compliance purposes in regulated industries.
  Fill this form to sign-up for private preview.

Features that are retired

[Retired] Pod Security Policy is being deprecated with AKS 1.25 and 2023-06-01 API version
Pod Security Policy (preview) is removed in AKS version 1.25 and retired on 1st August 2023 with AKS version 1.24 retirement. An AKS cluster with Pod Security Policy (preview) enabled can not be upgraded to AKS version 1.25+.
Required action We recommend you migrate to pod security admission controller or Azure policy to stay within Azure support. Pod Security Admission is a built-in policy solution for single cluster implementations. If you are looking for enterprise-grade policy, then Azure policy is a better choice. You can also disable Pod Security Policy (preview) on existing clusters to upgrade to AKS version 1.25+