Migrating from Azure Container Service (ACS) to Azure Kubernetes Service (AKS)
The goal of this document is to help you plan and execute a successful migration between Azure Container Service with Kubernetes (ACS) and Azure Kubernetes Service (AKS). This guide details the differences between ACS and AKS, provides an overview of the migration process, and should help you make key decisions.
Differences between ACS and AKS
ACS and AKS differ in some key areas that impact migration. You should review and plan to address the following differences before any migration.
- AKS nodes use Managed Disks
- Unmanaged disks will need to be converted before they can be attached to AKS nodes
StorageClassobjects for Azure disks will need to be changed from
PersistentVolumeswill need to use
- AKS currently supports only one agent pool
- Windows Server-based nodes are currently in private preview
- Check the list of AKS supported regions
- AKS is a managed service with a hosted Kubernetes control plane. You may need to modify your applications if you've previously modified the configuration of your ACS masters
Differences between Kubernetes versions
If you're migrating to a newer version of Kubernetes (ex: 1.7.x to 1.9.x), there are a few changes to the k8s API that will require your attention.
- Migrate a ThirdPartyResource to CustomResourceDefinition
- Workloads API changes in versions 1.8 and 1.9.
While AKS manages the Kubernetes control plane, you still define the size and number of nodes you want to include in your new cluster. Assuming you want a 1:1 mapping from ACS to AKS, you'll want to capture your existing ACS node information. You'll use this data when creating your new AKS cluster.
|Name||Count||VM Size||Operating System|
Because additional virtual machines will be deployed into your subscription during migration, you should verify that your quotas and limits are sufficient for these resources. You can learn more by reviewing Azure subscription and service limits. To check your current quotas, go to the subscriptions blade in the Azure portal, select your subscription, then select
Usage + quotas.
For complex applications, you'll typically migrate over time rather than all at once. That means that the old and new environments may need to communicate over the network. Applications that were previously able to use
ClusterIP services to communicate may need to be exposed as type
LoadBalancer and secured appropriately.
To complete the migration, you'll want to point clients to the new services running on AKS. The recommended way to redirect traffic is by updating DNS to point to the Load Balancer that sits in front of your AKS cluster.
Stateless application migration is the most straightforward case. You'll apply your YAML definitions to the new cluster, validate that everything is working as expected, and redirect traffic to make your new cluster active.
Migrating stateful applications requires careful planning to avoid data loss or unexpected downtime.
Highly Available Applications
Some stateful applications can be deployed in a high availability configuration and can copy data across replicas. If this describes your current deployment, it may be possible to create a new member on the new AKS cluster, and migrate with minimal impact to downstream callers. The migration steps for this scenario generally are:
- Create a new secondary replica on AKS
- Wait for data to replicate
- Fail over to make secondary replica the new primary
- Point traffic to the AKS cluster
Migrating Persistent Volumes
There are several factors to consider if you're migrating existing Persistent Volumes to AKS. Generally, the steps involved are:
- (Optional) Quiesce writes to the application (requires downtime)
- Snapshot disks
- Create new Managed Disks from snapshots
- Create Persistent Volumes in AKS
- Update Pod specifications to use existing volumes rather than PersistentVolumeClaims (static provisioning)
- Deploy application to AKS
- Point traffic to the AKS cluster
Important: If you choose not to quiesce writes, you'll need to replicate data to the new deployment, as you'll be missing data that was written since the disk snapshot
Open-source tools exist that can help you create Managed Disks and migrate volumes between Kubernetes clusters.
- noelbundick/azure-cli-disk-extension - copy and convert disks across Resource Groups and Azure regions
- yaron2/azure-kube-cli - enumerate ACS Kubernetes volumes and migrate them to an AKS cluster
Unlike disks, Azure Files can be mounted to multiple hosts concurrently. Neither Azure nor Kubernetes prevents you from creating a Pod in your AKS cluster that is still being used by your ACS cluster. To prevent data loss and unexpected behavior, you should ensure that both clusters aren't writing to the same files at the same time.
If your application can host multiple replicas pointing to the same file share, you can follow the stateless migration steps and deploy your YAML definitions to your new cluster.
If not, one possible migration approach involves the following steps:
- Deploy your application to AKS with a replica count of 0
- Scale the application on ACS to 0 (requires downtime)
- Scale the application on AKS up to 1
- Point traffic to the AKS cluster
In cases where you'd like to start with an empty share, then make a copy of the source data, you can use the
az storage file copy commands to migrate your data.
The recommended method is to use your existing CI/CD pipeline to deploy a known-good configuration to AKS. You'll clone your existing deploy tasks, and ensure that your
kubeconfig points to the new AKS cluster.
In cases where that's not possible, you'll need to export resource definition from ACS, and then apply them to AKS. You can use
kubectl to export objects.
kubectl get deployment -o=yaml --export > deployments.yaml
There are also several open-source tools that can help, depending on your needs:
1. Create an AKS cluster
You can follow the docs to create an AKS cluster via the Azure portal, Azure CLI, or Resource Manager template.
You can find sample Azure Resource Manager templates for AKS at the Azure/AKS repository on GitHub
2. Modify applications
Make any necessary modifications to your YAML definitions. Ex: replacing
3. (Optional) Migrate volumes
Migrate volumes from your ACS cluster to your AKS cluster. More details can be found in the Migrating Persistent Volumes section.
4. Deploy applications
Use your CI/CD system to deploy applications to AKS or use kubectl to apply the YAML definitions.
Validate that your applications are working as expected and that any migrated data has been copied over.
6. Redirect traffic
Update DNS to point clients to your AKS deployment.
7. Post-migration tasks
If you migrated volumes and chose not to quiesce writes, you'll need to copy that data to the new cluster.
We'd love to hear your thoughts. Choose the type you'd like to provide:
Our feedback system is built on GitHub Issues. Read more on our blog.