Tutorial: Integrate your Jenkins CI jobs with VSTS CD to Azure

Note

Build and release pipelines are called definitions in TFS 2018 and in older versions. Service connections are called service endpoints in TFS 2018 and in older versions.

Visual Studio Team Services (VSTS) provides integration with Jenkins so that you can use Jenkins for Continuous Integration (CI) while gaining several DevOps benefits from a VSTS release pipeline to Azure such as:

  • Reusing your existing investments in Jenkins Build jobs
  • Tracking work items and related code changes
  • View the entire CI/CD pipeline from a central location
  • Deploy various Azure services consistently via VSTS
  • Enforce quality of builds to gate deployments
  • Define work flows such as manual approval processes and CI triggers

In this tutorial, you use Jenkins to build a Java web app that uses Maven and VSTS or Team Foundation Server (TFS) to deploy to an Azure App Service.

You will:

  • Get the Sample App
  • Configure Jenkins credentials and the VSTS plugin
  • Configure a Jenkins CI build with VSTS integration
  • Create a Jenkins service connection and Service Hooks in VSTS
  • Create a VSTS Release Pipeline for CD to Azure
  • Test the CI/CD pipeline with a pull request
  • (Optionally) Create a VSTS build pipeline to wrap the Jenkins CI job

Before you begin

Get the sample app

You need an app stored in a Git repository. You use this app to build and deploy. For this tutorial, we recommend you use this sample Java app available from GitHub. This tutorial uses a Java and Maven sample application that is configured for deployment to Azure App Service. If you want to work with your own repository, you should configure a similar sample.

  1. In VSTS, on the Code hub for your VSTS project, select the option to Import repository.

  2. In the Import a Git repository dialog box, paste the following URL into the Clone URL text box.

    https://github.com/Azure-Samples/app-service-maven
    
  3. Click Import to copy the sample code into your Git repo.

  4. Select Clone at the top right, and keep the clone url for future steps in this tutorial.

Configure Jenkins credentials and the VSTS plugin

You must configure credentials for connecting to VSTS, and a plug in for VS Team Services Continuous Deployment on your Jenkins server. When using credentials to connect to VSTS, it is a best practice to use a personal access token (PAT). You also need to create a Jenkins credential to use in your Jenkins build jobs.

Note

Ensure the PAT you use for the following steps contains the Release (read, write, execute and manage), Code (read), Build (read and execute) permissions in VSTS.

  1. Create a PAT in your VSTS account. Jenkins requires this information to access your VSTS organization. Ensure you store the token information for upcoming steps in this section. Read How do I create a personal access token for VSTS and TFS to learn how to generate a PAT, or use an existing PAT if you have one.

  2. Open your Jenkins account and select Credentials, System, and then choose Add Domain.

  3. Enter a name, description, and then select Add hostname.

  4. In the Include text box, enter \*.visualstudio.com to enable this credential for all VSTS organizations. You can optionally enter your specific VSTS organization.

  5. Select Ok. You will see a message about empty credentials. Select the link for adding some credentials.

  6. For Kind, choose Username with password. Enter the Username and the PAT you created earlier for Password. Select Ok.

  7. Navigate back to the Jenkins home page.

  8. Choose Manage Jenkins.

  9. In the Manage Jenkins page, choose Manage Plugins.

  10. Filter the available list to find the VS Team Services Continuous Deployment plug in and choose the Install without restart option. You can find additional information about the VSTS plugin here.

Configure a Jenkins CI build with VSTS integration

You create a Jenkins build job to use the source code stored in your VSTS repository and execute a basic build. Ensure your Jenkins server has Maven installed and configured. You also create triggers to enable CI builds when code changes are pushed to the repository, and a CD trigger to initiate a VSTS release after the Jenkins build completes via a post build action.

  1. Navigate to your Jenkins server. Click New Item. Enter an item name.

  2. Choose Freestyle project. Select OK.

  3. In the Source Code Management tab, select Git and enter the clone url you saved earlier for the VSTS Repository URL and the branch containing your app code. If you are using Team Foundation Server, you can choose the option for Team Foundation Version Control (TFVC).
    Add a repo to your build

  4. Select the Credentials drop down and choose the credential you created earlier. You should successfully authenticate to your VSTS repository and not receive errors before continuing. If you see errors, you likely have an issue with your credentials and VSTS PAT.

  5. Select the Build Triggers tab, then tick the check boxes for Build when a change is pushed to TFS/Team Services and Build when a change is pushed to a TFS pull request. These two options rely on VSTS Service Hooks which you create in the next section.

  6. Select the Build tab, Add build step, and then choose Invoke top-level Maven targets.

  7. Select Advanced.... Enter clean package for Goals, and then enter pom.xml for POM.

  8. Select the Post-build Actions tab. Choose Add post-build action, then select Archive the artifacts.

  9. For Files to archive enter the following:

    target/*.war
    
  10. Select the Post-build Actions tab. Choose Add post-build action, then select Trigger release in TFS/Team Services.

  11. Enter a collection URL. An example is http://{YourVSTSAccount}.visualstudio.com/DefaultCollection

  12. Enter the project and choose a release pipeline name. Store the release pipeline name since it is needed in later steps of this tutorial.

  13. Enter the username and PAT you created earlier.

  14. Save the Jenkins project.

Create a Jenkins service connection and Service Hooks in VSTS

You configure a Jenkins service connection to allow VSTS to connect to your Jenkins server. You must also configure two Jenkins service hooks so you can execute CI builds via automated triggers for both simple commits as well as pull requests to your VSTS Git repository.

  1. Open the Services page in VSTS, open the New service connection list, and choose Jenkins. Add a Jenkins endpoint

  2. Enter a name for the connection.

  3. Enter the URL of your Jenkins server, and if using http tick the Accept untrusted SSL certificates option. An example URL is: http://{YourJenkinsURL}.westcentralus.cloudapp.azure.com

  4. Enter the user name and password for your Jenkins account.

  5. Choose Verify connection to check that the information is correct.

  6. Choose OK to create the service connection.

  7. From the Service Hooks page in VSTS, Select the + to add a new service and choose Jenkins. Select next.

  8. Choose Code pushed for the type of event trigger. Select your code repository that contains the sample application you imported earlier. Select Next.

  9. Enter the URL of your Jenkins server. An example is below. http://{YourJenkinsURL}.westcentralus.cloudapp.azure.com

  10. Enter the user name and password for your Jenkins account.

  11. Select the Jenkins build you created earlier.

  12. Choose Test to check that the information is correct.

  13. Choose Finish to create the service connection.

  14. Repeat the steps in this section to create another service hook for the pull request merge attempted trigger type. This will allow either simple commits to the repository as well as pull requests to both trigger the Jenkins build.

Create a VSTS release pipeline for CD to Azure

A release pipeline specifies the process VSTS executes to deploy the app. In this example, you deploy your app that originates from the Jenkins CI system. You deploy to an Azure App Service running Tomcat and Java.

To create the release pipeline in VSTS:

  1. Open Releases on the Build & Release hub in VSTS, and choose Create release pipeline.

  2. Select the Empty template by choosing Start with an Empty process.

  3. In the Artifacts section, click on + Add Artifact and choose Jenkins for Source type. Select your Jenkins service connection. Then select the Jenkins source job and choose Add.

  4. Next to the 1 phase, 0 environments link, select the + on the Agent Phase to add a task to the phase.

  5. Search for the Azure App Service Deploy task. Select Add to add the task.

  6. Choose your Azure Subscription. If you do not have an existing Azure connection in VSTS, you can follow the steps here to create one.

  7. Enter a name for your existing Web App for the App Service name. This name must match your existing Web App name from the prerequisites.

  8. For the Package or folder setting, select the ellipsis to browse, and then select your .war artifact.

  9. Ensure the name for your release pipeline matches the same name you chose earlier during the Create a Jenkins service connection and Service Hooks in VSTS steps.

  10. Click Save, and then click OK to save the release pipeline.

Test the CI/CD pipeline with a pull request

You can initiate the CI build and the subsequent CD deployment to Azure by completing a pull request into your master branch. The Jenkins build will initiate due to the service hook you set up earlier, and the Jenkins post build action will initiate a VSTS release which will deploy your app to the Azure App Service.

  1. Navigate to Code in VSTS, then select your repository.

  2. Select New branch to create a branch from the master branch. Enter a name for your new branch, and then select Create branch.

  3. Select your new branch, then navigate to the src/main/webapp/index.jsp file.

  4. Select Edit, then make a change to the Title for the index.jsp page, and then Commit your changes.

  5. Select Pull Requests, then select New Pull Request. Select Create to issue a pull request from your branch to master.

  6. Select Approve, then select Complete, then Complete Merge. This code change will initiate a CI build in Jenkins.

  7. Navigate to your Jenkins dashboard and examine the build that is executing. Once it finishes, you can navigate to VSTS to watch the Release Pipeline execute. In a few moments your latest changes will be deployed to the Azure App service.

You are now using Jenkins CI builds with a VSTS code repository and release pipeline to perform CI/CD to Azure App Service. You can easily track your code changes and deployments via the rich reporting capabilities of VSTS, and leverage Jenkins to execute CI builds.

(Optionally) Create a VSTS build pipeline to wrap the Jenkins CI job

There is an additional approach (pattern) possible when integrating Jenkins and VSTS CI/CD. You can optionally create a VSTS build pipeline to wrap the Jenkins build job, which will help you monitor the Jenkins job from VSTS. This pattern works slightly differently by using the built-in VSTS build capabilities to trigger the pipeline when new code is pushed to the repository. The approach for using both VSTS build and Jenkins together requires both VSTS build and Jenkins agents. To enable this pattern, you will modify a few items for the VSTS and Jenkins CI/CD pipeline that you created in earlier steps.

  1. Navigate to the Jenkins build job you created earlier.

  2. Select the Build Triggers tab, then remove the tick for the check boxes for Build when a change is pushed to TFS/Team Services and Build when a change is pushed to a TFS pull request. These two options are not needed for this pattern since the VSTS build pipeline has the ability to trigger the CI/CD pipeline when new code is pushed to the repo.

  3. Navigate to your VSTS organization.

  4. Open Builds on the Build & Release hub in VSTS, and choose + New to create a new build pipeline.

  5. Ensure VSTS Git is selected for the source, and then select Continue.

  6. Search for the Jenkins template, and then select Apply.

  7. Select Process, and then select Hosted VS 2017 for the Agent queue.

  8. For the Job name parameter, enter the Jenkins Job name you created in the earlier steps of this tutorial. This name must match exactly. This will allow you to queue the Jenkins job and download the artifacts it produces.

  9. Select the Jenkins service connection.

  10. Select Get sources, and then check the Don't sync sources option. Since the Jenkins job is handling fetching the sources from VSTS, we do not need the VSTS build pipeline to perform this step. However, this step will configure a local working directory on the VSTS build agent.

  11. Ensure the Report build status option is enabled so the build status will be reported on the code tab for the VSTS repository.

  12. Select the Jenkins build tasks under Phase 1, and then examine the additional options provided by the tasks. There is a task to queue the Jenkins job, download the artifacts produced by the job, and then finally to publish the artifacts which can later be used by the VSTS release pipeline.

  13. Select the Queue Jenkins Job task. Capture console output and wait for completion is enabled to help provide the Jenkins logging information to VSTS, and also it will cause the VSTS build to wait for this step to complete successfully in Jenkins before proceeding. This causes the VSTS build to succeed for fail based on the Jenkins result. Capture pipeline output and wait for completion is similarly enabled to ensure VSTS succeeds or fails based on the entire Jenkins pipeline result. These two options help you view Jenkins logging information in a single place (VSTS) for the entire CI/CD pipeline.

  14. Open Releases on the Build & Release hub in VSTS, and choose the release pipeline you created earlier. Select the ellipsis and choose Edit.

  15. Delete the artifact you used earlier. Add a new artifact by choosing your VSTS build pipeline. The VSTS build pipeline will now provide the artifact for the CD part of the pipeline.

  16. In the Artifacts section, click on + Add Artifact and choose Jenkins for Source type. Select your Jenkins service connection. Then select the Jenkins source job and choose Add.

You can now test the pipeline (as you did on earlier steps) with a pull request or by pushing new code to your branch. A VSTS build will initiate, a Jenkins job executes, and a VSTS release pipeline will deploy your changes to Azure.

Next Steps

In this tutorial, you automated the deployment of an app to Azure using Jenkins for the CI build and VSTS for release. You learned how to:

  • Get the Sample App
  • Configure Jenkins credentials and VSTS plugin
  • Configure a Jenkins CI build with VSTS integration
  • Create a Jenkins service connection and service hooks in VSTS
  • Create a VSTS Release Pipeline for CD to Azure
  • Test the CI/CD pipeline with a pull request
  • (Optionally) Create a VSTS build pipeline to wrap the Jenkins CI job