您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

使用 CI/CD 工具自动执行 Azure 流分析作业的生成、测试和部署Automate builds, tests, and deployments of an Azure Stream Analytics job using CI/CD tools

可以使用 Azure 流分析 CI/CD npm 包自动生成、测试和部署 Azure 流分析 Visual Studio Code 或 Visual Studio 项目。You can use the Azure Stream Analytics CI/CD npm package to automatically build, test, and deploy your Azure Stream Analytics Visual Studio Code or Visual Studio projects. 可以使用开发工具创建项目,也可以从现有的流分析作业中导出这些项目。The projects can be created using development tools or they can be exported from existing Stream Analytics jobs. 本文介绍了如何将 npm 包与任何 CI/CD 系统一起使用。This article describes how to use the npm package with any CI/CD system. 有关 Azure Pipelines 的部署,请参阅 使用 Azure DevOps 为流分析作业创建 CI/CD 管道For deployment with Azure Pipelines, see Use Azure DevOps to create a CI/CD pipeline for a Stream Analytics job.

安装Installation

可以直接 下载包 ,也可以使用命令 全局 安装包 npm install -g azure-streamanalytics-cicdYou can download the package directly, or install it globally using the npm install -g azure-streamanalytics-cicd command. 建议使用命令,该命令还可以在 Azure Pipelines 的生成管道的 PowerShell 或 Azure CLI 脚本任务中使用。We recommend using the command, which can also be used in a PowerShell or Azure CLI script task of a build pipeline in Azure Pipelines.

生成项目Build the project

Streamanalytics-default-central-us-cicd npm 包提供了用于生成流分析 Visual Studio Code 项目Visual Studio 项目的 Azure 资源管理器模板的工具。The asa-streamanalytics-cicd npm package provides the tools to generate Azure Resource Manager templates of Stream Analytics Visual Studio Code projects or Visual Studio projects. 你还可以在 Windows、macOS 和 Linux 上使用 npm 包,而无需安装 Visual Studio Code 或 Visual Studio。You can also use the npm package on Windows, macOS, and Linux without installing Visual Studio Code or Visual Studio.

安装程序包后,请使用以下命令生成流分析项目。Once you have installed the package, use the following command to build your Stream Analytics projects.

azure-streamanalytics-cicd build -project <projectFullPath> [-outputPath <outputPath>]

" 生成 " 命令执行关键字语法检查并输出 Azure 资源管理器模板。The build command does a keyword syntax check and outputs the Azure Resource Manager template.

参数Parameter 说明Description
-project Visual Studio Code 项目的文件的绝对 asaproj.js 路径,或 [项目名称]. Asaproj for Visual Studio 项目。The absolute path of the asaproj.json file for your Visual Studio Code project or [Your project name].asaproj for Visual Studio project.
-outputPath Azure 资源管理器模板的输出文件夹的路径。The path of the output folder for Azure Resource Manager Templates. 如果未指定此项,则将模板放在当前目录中。If it is not specified, the templates will be placed in the current directory.
azure-streamanalytics-cicd build -project "/Users/username/projects/samplejob/asaproj.json"

当流分析项目成功生成时,它会在 output 文件夹下生成以下两个文件:When a Stream Analytics project builds successfully, it generates the following two files under the output folder:

  • Azure 资源管理器模板文件Azure Resource Manager template file

    [ProjectName].JobTemplate.json

  • Azure 资源管理器参数文件Azure Resource Manager parameter file

    [ProjectName].JobTemplate.parameters.json

文件 parameters.js上的默认参数来自 Visual Studio Code 或 Visual Studio 项目中的设置。The default parameters in the parameters.json file are from the settings in your Visual Studio Code or Visual Studio project. 如果要部署到其他环境,请相应地替换参数。If you want to deploy to another environment, replace the parameters accordingly.

所有凭据的默认值均为 nullThe default values for all credentials are null. 在部署到 Azure 之前,需要设置这些值。You are required to set the values before you deploy to Azure.

"Input_EntryStream_sharedAccessPolicyKey": {
      "value": null
    },

若要将托管标识用于 Azure Data Lake Store Gen1 作为输出接收器,需要在部署到 Azure 之前,使用 PowerShell 提供对服务主体的访问权限。To use Managed Identity for Azure Data Lake Store Gen1 as an output sink, you need to provide access to the service principal using PowerShell before you deploy to Azure. 了解有关如何使用资源管理器模板部署具有托管标识的 ADLS Gen1 的详细信息。Learn more about how to deploy ADLS Gen1 with Managed Identity with Resource Manager template.

本地运行Local run

如果你的项目已指定本地输入文件,则可使用命令在本地运行流分析脚本 localrunIf your project has specified local input files, you can run a Stream Analytics script locally by using the localrun command.

azure-streamanalytics-cicd localrun -project <projectFullPath> [-outputPath <outputPath>] [-customCodeZipFilePath <zipFilePath>]
参数Parameter 说明Description
-project Visual Studio Code 项目的 asaproj.js 文件路径,或 [项目名称]. Asaproj for Visual Studio 项目。The path of the asaproj.json file for your Visual Studio Code project or [Your project name].asaproj for Visual Studio project.
-outputPath 输出文件夹的路径。The path of the output folder. 如果未指定,则输出结果文件将放在当前目录中。If it is not specified, the output result files will be placed in the current directory.
-customCodeZipFilePath C # 自定义代码(如 UDF 或反序列化程序)的 zip 文件的路径(如果使用)。The path of the zip file for C# custom code, such as a UDF or deserializer, if they are used. 将 Dll 打包到一个 zip 文件中,并指定此路径。Package the DLLs into a zip file and specify this path.
azure-streamanalytics-cicd localrun -project "/Users/roger/projects/samplejob/asaproj.json"

备注

JavaScript UDF 仅适用于 Windows。JavaScript UDF only works on Windows.

自动测试Automated test

可以使用 CI/CD npm 包来配置和运行流分析脚本的自动测试。You can use the CI/CD npm package to configure and run automated tests for your Stream Analytics script.

添加测试用例Add a test case

测试用例在测试配置文件中进行了介绍。The test cases are described in a test configuration file. 若要开始,请使用 addtestcase 命令将测试用例模板添加到测试配置文件中。To get started, use the addtestcase command to add a test case template to the test configuration file. 如果测试配置文件不存在,则默认情况下会创建一个。If the test configuration file doesn't exist, one is created by default.

azure-streamanalytics-cicd addtestcase -project <projectFullPath> [-testConfigPath <testConfigFileFullPath>]
参数Parameter 说明Description
-project Visual Studio Code 项目的 asaproj.js 文件路径,或 [项目名称]. Asaproj for Visual Studio 项目。The path of the asaproj.json file for your Visual Studio Code project or [Your project name].asaproj for Visual Studio project.
-testConfigPath 测试配置文件的路径。The path of the test configuration file. 如果未指定此值,则会在 \test 文件的当前 asaproj.js 目录下的文件中搜索文件,默认文件名 testConfig.js为 onIf it is not specified, the file will be searched in \test under the current directory of the asaproj.json file, with default file name testConfig.json. 如果不存在,则将创建新的文件。A new file will be created if not existed.

备注

Script生成的 testConfig.js 文件中的值仅用于提供上下文;它不用于测试逻辑。The Script value in the generated testConfig.json file is only for providing the context; It's not used in the testing logic.

azure-streamanalytics-cicd addtestcase -project "/Users/roger/projects/samplejob/asaproj.json"

如果测试配置文件为空,则会将以下内容写入文件中。If the test configuration file is empty, the following content is written into the file. 否则,会将一个测试用例添加到 测试用例 的数组中。Otherwise, a test case is added into the array of TestCases. 必要的输入配置会根据输入配置文件(如果存在)自动填充。Necessary input configurations are automatically filled according to the input configuration files, if they exist. 否则,将配置默认值。Otherwise, default values are configured. 在运行测试之前,必须指定每个输入和预期输出的 FilePathFilePath of each input and expected output must be specified before running the test. 你可以手动修改配置。You can modify the configuration manually.

如果希望测试验证忽略特定输出,请将预期输出的 必填 字段设置为 falseIf you want the test validation to ignore a certain output, set the Required field of that expected output to false.

{
  "Script": "",
  "TestCases": [
    {
      "Name": "Case 1",
      "Inputs": [
        {
          "InputAlias": [Input alias string],
          "Type": "Data Stream",
          "Format": "JSON",
          "FilePath": [Required],
          "ScriptType": "InputMock"
        }
      ],
      "ExpectedOutputs": [
        {
          "OutputAlias": [Output alias string],
          "FilePath": [Required],
          "Required": true
        }
      ]
    }
  ]
}

备注

目前,该元素唯一允许的值 ScriptTypeInputMock ,这也是默认值。Currently, the only allowed value for the ScriptType element is InputMock, which is also the default value. 如果将其设置为任何其他值,将忽略该值,并使用默认值 (InputMock) 。If you set it to any other value, it's ignored and the default value (InputMock) is used.

运行单元测试Run a unit test

可以使用以下命令为项目运行多个测试用例。You can use the following command to run multiple test cases for your project. 在输出文件夹中生成测试结果的摘要。A summary of test results is generated in the output folder. 对于所有已传递的测试,进程退出,代码为 0 ;如果发生异常,则为 -1 ; -2 个测试失败。The process exits with code 0 for all tests passed; -1 for exception occurred; -2 for tests failed.

azure-streamanalytics-cicd test -project <projectFullPath> [-testConfigPath <testConfigFileFullPath>] [-outputPath <outputPath>] [-customCodeZipFilePath <zipFilePath>]
参数Parameter 说明Description
-project Visual Studio Code 项目的 asaproj.js 文件路径,或 [项目名称]. Asaproj for Visual Studio 项目。The path of the asaproj.json file for your Visual Studio Code project or [Your project name].asaproj for Visual Studio project.
-testConfigPath 测试配置文件的路径。The path to the test configuration file. 如果未指定此值,则会在 \test 文件的当前 asaproj.js 目录下的文件中搜索文件,默认文件名 testConfig.js为 onIf it is not specified, the file will be searched in \test under the current directory of the asaproj.json file, with default file name testConfig.json.
-outputPath 测试结果输出文件夹的路径。The path of the test result output folder. 如果未指定,则输出结果文件将放在当前目录中。If it is not specified, the output result files will be placed in the current directory.
-customCodeZipFilePath 自定义代码(如 UDF 或反序列化程序)的 zip 文件的路径(如果使用)。The path of the zip file for custom code such as a UDF or deserializer, if they are used.

完成所有测试后,将在输出文件夹中生成 JSON 格式的测试结果的摘要。When all tests are finished, a summary of the test results in JSON format is generated in the output folder. 摘要文件的名称为 testResultSummary.jsThe summary file is named testResultSummary.json.

{
  "Total": (integer) total_number_of_test_cases,
  "Passed": (integer) number_of_passed_test_cases,
  "Failed": (integer) number_of_failed_test_cases,
  "Script": (string) absolute_path_to_asaql_file,
  "Results": [ (array) detailed_results_of_test_cases
    {
      "Name": (string) name_of_test_case,
      "Status": (integer) 0(passed)_or_1(failed),
      "Time": (string) time_span_of_running_test_case,
      "OutputMatched": [ (array) records_of_actual_outputs_equal_to_expected_outputs
        {
          "OutputAlias": (string) output_alias,
          "ExpectedOutput": (string) path_to_the_expected_output_file,
          "Output": (string) path_to_the_actual_output_file
        }
      ],
      "OutputNotEqual": [ (array) records_of_actual_outputs_not_equal_to_expected_outputs
        {
          "OutputAlias": (string) output_alias,
          "ExpectedOutput": (string) path_to_the_expected_output_file,
          "Output": (string) path_to_the_actual_output_file
        }
      ],
      "OutputMissing": [ (array) records_of_actual_outputs_missing
        {
          "OutputAlias": (string) output_alias,
          "ExpectedOutput": (string) path_to_the_expected_output_file,
          "Output": ""
        }
      ],
      "OutputUnexpected": [ (array) records_of_actual_outputs_unexpected
        {
          "OutputAlias": (string) output_alias,
          "ExpectedOutput": "",
          "Output": (string) path_to_the_actual_output_file
        }
      ],
      "OutputUnrequired": [ (array) records_of_actual_outputs_unrequired_to_be_checked
        {
          "OutputAlias": (string) output_alias,
          "ExpectedOutput": (string) path_to_the_expected_output_file,
          "Output": (string) path_to_the_actual_output_file
        }
      ]
    }
  ],
  "Time": (string) time_span_of_running_all_test_cases,
}

部署到 AzureDeploy to Azure

可以使用 "生成" 中生成的 Azure 资源管理器模板和参数文件将 作业部署到 AzureYou can use the Azure Resource Manager template and parameter files generated from Build to deploy your job to Azure.

后续步骤Next steps