(DEPRECATED) Full CI/CD pipeline to deploy a multi-container application on Azure Container Service with Docker Swarm using Azure DevOps Services
The Azure Container Service (ACS) is being deprecated. No new features or functionality are being added to ACS. All of the APIs, portal experience, CLI commands and documentation are marked as deprecated.
For more information, see the Azure Container Service deprecation announcement on Azure.com.
We recommend that you deploy one of the following Azure Marketplace solutions:
Docker EE for Azure
If you want to use Kubernetes, see Azure Kubernetes Service.
One of the biggest challenges when developing modern applications for the cloud is being able to deliver these applications continuously. In this article, you learn how to implement a full continuous integration and deployment (CI/CD) pipeline using Azure Container Service with Docker Swarm, Azure Container Registry, and Azure Pipelines management.
This article is based on a simple application, available on GitHub, developed with ASP.NET Core. The application is composed of four different services: three web APIs and one web front end:
The objective is to deliver this application continuously in a Docker Swarm cluster, using Azure DevOps Services. The following figure details this continuous delivery pipeline:
Here is a brief explanation of the steps:
- Code changes are committed to the source code repository (here, GitHub)
- GitHub triggers a build in Azure DevOps Services
- Azure DevOps Services gets the latest version of the sources and builds all the images that compose the application
- Azure DevOps Services pushes each image to a Docker registry created using the Azure Container Registry service
- Azure DevOps Services triggers a new release
- The release runs some commands using SSH on the Azure container service cluster master node
- Docker Swarm on the cluster pulls the latest version of the images
- The new version of the application is deployed using Docker Compose
Before starting this tutorial, you need to complete the following tasks:
- Create a Swarm cluster in Azure Container Service
- Connect with the Swarm cluster in Azure Container Service
- Create an Azure container registry
- Have an Azure DevOps Services organization and project created
- Fork the GitHub repository to your GitHub account
The Docker Swarm orchestrator in Azure Container Service uses legacy standalone Swarm. Currently, the integrated Swarm mode (in Docker 1.12 and higher) is not a supported orchestrator in Azure Container Service. If you want to deploy a Swarm mode cluster in Azure, use the open-source ACS Engine, a community-contributed quickstart template, or a Docker solution in the Azure Marketplace.
You also need an Ubuntu (14.04 or 16.04) machine with Docker installed. This machine is used by Azure DevOps Services during the Azure Pipelines processes. One way to create this machine is to use the image available in the Azure Marketplace.
Step 1: Configure your Azure DevOps Services organization
In this section, you configure your Azure DevOps Services organization.
Configure an Azure DevOps Services Linux build agent
To create Docker images and push these images into an Azure container registry from an Azure DevOps Services build, you need to register a Linux agent. You have these installation options:
Install the Docker Integration Azure DevOps Services extension
Microsoft provides an Azure DevOps Services extension to work with Docker in Azure Pipelines processes. This extension is available in the Azure DevOps Services Marketplace. Click Install to add this extension to your Azure DevOps Services organization:
You are asked to connect to your Azure DevOps Services organization using your credentials.
Connect Azure DevOps Services and GitHub
Set up a connection between your Azure DevOps Services project and your GitHub account.
In your Azure DevOps Services project, click the Settings icon in the toolbar, and select Services.
On the left, click New Service Endpoint > GitHub.
To authorize Azure DevOps Services to work with your GitHub account, click Authorize and follow the procedure in the window that opens.
Connect Azure DevOps Services to your Azure container registry and Azure Container Service cluster
The last steps before getting into the CI/CD pipeline are to configure external connections to your container registry and your Docker Swarm cluster in Azure.
In the Services settings of your Azure DevOps Services project, add a service endpoint of type Docker Registry.
In the popup that opens, enter the URL and the credentials of your Azure container registry.
For the Docker Swarm cluster, add an endpoint of type SSH. Then enter the SSH connection information of your Swarm cluster.
All the configuration is done now. In the next steps, you create the CI/CD pipeline that builds and deploys the application to the Docker Swarm cluster.
Step 2: Create the build pipeline
In this step, you set up a build pipeline for your Azure DevOps Services project and define the build workflow for your container images
Initial pipeline setup
To create a build pipeline, connect to your Azure DevOps Services project and click Build & Release.
In the Build definitions section, click + New. Select the Empty template.
Configure the new build with a GitHub repository source, check Continuous integration, and select the agent queue where you registered your Linux agent. Click Create to create the build pipeline.
On the Build Definitions page, first open the Repository tab and configure the build to use the fork of the MyShop project that you created in the prerequisites. Make sure that you select acs-docs as the Default branch.
On the Triggers tab, configure the build to be triggered after each commit. Select Continuous integration and Batch changes.
Define the build workflow
The next steps define the build workflow. There are five container images to build for the MyShop application. Each image is built using the Dockerfile located in the project folders:
You need to add two Docker steps for each image, one to build the image, and one to push the image in the Azure container registry.
To add a step in the build workflow, click + Add build step and select Docker.
For each image, configure one step that uses the
For the build operation, select your Azure container registry, the Build an image action, and the Dockerfile that defines each image. Set the Build context as the Dockerfile root directory, and define the Image Name.
As shown on the preceding screen, start the image name with the URI of your Azure container registry. (You can also use a build variable to parameterize the tag of the image, such as the build identifier in this example.)
For each image, configure a second step that uses the
For the push operation, select your Azure container registry, the Push an image action, and enter the Image Name that is built in the previous step.
After you configure the build and push steps for each of the five images, add two more steps in the build workflow.
a. A command-line task that uses a bash script to replace the BuildNumber occurrence in the docker-compose.yml file with the current build Id. See the following screen for details.
b. A task that drops the updated Compose file as a build artifact so it can be used in the release. See the following screen for details.
Click Save and name your build pipeline.
Step 3: Create the release pipeline
Azure DevOps Services allows you to manage releases across environments. You can enable continuous deployment to make sure that your application is deployed on your different environments (such as dev, test, pre-production, and production) in a smooth way. You can create a new environment that represents your Azure Container Service Docker Swarm cluster.
Initial release setup
To create a release pipeline, click Releases > + Release
To configure the artifact source, Click Artifacts > Link an artifact source. Here, link this new release pipeline to the build that you defined in the previous step. By doing this, the docker-compose.yml file is available in the release process.
To configure the release trigger, click Triggers and select Continuous Deployment. Set the trigger on the same artifact source. This setting ensures that a new release starts as soon as the build completes successfully.
Define the release workflow
The release workflow is composed of two tasks that you add.
Configure a task to securely copy the compose file to a deploy folder on the Docker Swarm master node, using the SSH connection you configured previously. See the following screen for details.
Configure a second task to execute a bash command to run
docker-composecommands on the master node. See the following screen for details.
The command executed on the master use the Docker CLI and the Docker-Compose CLI to do the following tasks:
- Login to the Azure container registry (it uses three build variab`les that are defined in the Variables tab)
- Define the DOCKER_HOST variable to work with the Swarm endpoint (:2375)
- Navigate to the deploy folder that was created by the preceding secure copy task and that contains the docker-compose.yml file
docker-composecommands that pull the new images, stop the services, remove the services, and create the containers.
As shown on the preceding screen, leave the Fail on STDERR checkbox unchecked. This is an important setting, because
docker-composeprints several diagnostic messages, such as containers are stopping or being deleted, on the standard error output. If you check the checkbox, Azure DevOps Services reports that errors occurred during the release, even if all goes well.
- Save this new release pipeline.
This deployment includes some downtime because we are stopping the old services and running the new one. It is possible to avoid this by doing a blue-green deployment.
Step 4. Test the CI/CD pipeline
Now that you are done with the configuration, it's time to test this new CI/CD pipeline. The easiest way to test it is to update the source code and commit the changes into your GitHub repository. A few seconds after you push the code, you will see a new build running in Azure DevOps Services. Once completed successfully, a new release will be triggered and will deploy the new version of the application on the Azure Container Service cluster.
- For more information about CI/CD with Azure DevOps Services, see the Azure DevOps Services Build overview.