Azure DevOps Task for Azure Data Explorer

Azure DevOps Services provides development collaboration tools such as high-performance pipelines, free private Git repositories, configurable Kanban boards, and extensive automated and continuous testing capabilities. Azure Pipelines is an Azure DevOps capability that enables you to manage CI/CD to deploy your code with high-performance pipelines that work with any language, platform, and cloud. Azure Data Explorer - Admin Commands is the Azure Pipelines task that enables you to create release pipelines and deploy your database changes to your Azure Data Explorer databases. It's available for free in the Visual Studio Marketplace.

This document describes a simple example on the use of the Azure Data Explorer – Admin Commands task to deploy your schema changes to your database. For complete CI/CD pipelines, refer to Azure DevOps documentation.

Prerequisites

Create folders

Create the following sample folders (Functions, Policies, Tables) in your Git repository. Copy the files from here into the respective folders as seen below and commit the changes. The sample files are provided to execute the following workflow.

Create folders

Tip

When creating your own workflow, we recommend making your code idempotent. For example, use .create-merge table instead of .create table, and use .create-or-alter function instead of .create function.

Create a release pipeline

  1. Sign in to your Azure DevOps organization.

  2. Select Pipelines > Releases from left-hand menu and select New pipeline.

    New pipeline

  3. The New release pipeline window opens. In the Pipelines tab, in the Select a template pane, select Empty job.

    Select a template

  4. Select Stage button. In Stage pane, add the Stage name. Select Save to save your pipeline.

    Name the stage

  5. Select Add an artifact button. In the Add an artifact pane, select the repository where your code exists, fill out relevant information, and click Add. Select Save to save your pipeline.

    Add an artifact

  6. In the Variables tab, select + Add to create a variable for Endpoint URL that will be used in the task. Write the Name and the Value of the endpoint. Select Save to save your pipeline.

    Create variable

    To find your Endpoint_URL, the overview page of your Azure Data Explorer Cluster in the Azure portal contains the Azure Data Explorer cluster URI. Construct the URI in the following format https://<Azure Data Explorer cluster URI>?DatabaseName=<DBName>. For example, https://kustodocs.westus.kusto.windows.net?DatabaseName=SampleDB

    Azure Data Explorer cluster URI

Create tasks to deploy

  1. In the Pipeline tab, click on 1 job, 0 task to add tasks.

    Add tasks

  2. Create three tasks to deploy Tables, Functions, and Policies, in this order.

  3. In the Tasks tab, select + by Agent job. Search for Azure Data Explorer. In Marketplace, install the Azure Data Explorer – Admin Commands extension. Then, select Add in Run Azure Data Explorer Command.

    Add admin commands

  4. Click on Kusto Command on the left and update the task with the following information:

    • Display name: Name of the task
    • File path: In the Tables task, specify /Tables/.csl since the table creation files are in the Table folder.
    • Endpoint URL: enter the EndPoint URLvariable created in previous step.
    • Select Use Service Endpoint and select + New.

    Update Kusto command task

  5. Complete the following information in the Add Azure Data Explorer service connection window:

    Setting Suggested value
    Connection name Enter a name to identify this service endpoint
    Cluster Url Value can be found in the overview section of your Azure Data Explorer Cluster in the Azure portal
    Service Principal Id Enter the AAD App ID (created as prerequisite)
    Service Principal App Key Enter the AAD App Key (created as prerequisite)
    AAD tenant Id Enter your AAD tenant (such as microsoft.com, contoso.com...)

    Select Allow all pipelines to use this connection checkbox. Select OK.

    Add service connection

  6. Repeat steps 1-5 another two times to deploy files from the Functions and Policies folders. Select Save. In the Tasks tab, see the three tasks created: Deploy Tables, Deploy Functions, and Deploy Policies.

    Deploy all folders

  7. Select + Release > Create release to create a release.

    Create a release

  8. In the Logs tab, check the deployment status is successful.

    Deployment is successful

You have now completed creation of a release pipeline for deployment of three tasks to pre-production.