App Center Test task

Azure Pipelines | Azure DevOps Server 2019 | TFS 2018 | TFS 2017

This task lets you run test suites against an application binary (.apk or .ipa file) using App Center Test.

Note

In Microsoft Team Foundation Server (TFS) 2018 and previous versions, run and release pipelines are called definitions, runs are called builds, service connections are called service endpoints, stages are called environments, and jobs are called phases.

YAML snippet

# App Center test
# Test app packages with Visual Studio App Center
- task: AppCenterTest@1
  inputs:
    appFile: 
    #artifactsDirectory: '$(Build.ArtifactStagingDirectory)/AppCenterTest' 
    #prepareTests: true # Optional
    #frameworkOption: 'appium' # Required when prepareTests == True# Options: appium, espresso, calabash, uitest, xcuitest
    #appiumBuildDirectory: # Required when prepareTests == True && Framework == Appium
    #espressoBuildDirectory: # Optional
    #espressoTestApkFile: # Optional
    #calabashProjectDirectory: # Required when prepareTests == True && Framework == Calabash
    #calabashConfigFile: # Optional
    #calabashProfile: # Optional
    #calabashSkipConfigCheck: # Optional
    #uiTestBuildDirectory: # Required when prepareTests == True && Framework == Uitest
    #uitestStorePath: # Optional
    #uiTestStorePassword: # Optional
    #uitestKeyAlias: # Optional
    #uiTestKeyPassword: # Optional
    #uiTestToolsDirectory: # Optional
    #signInfo: # Optional
    #xcUITestBuildDirectory: # Optional
    #xcUITestIpaFile: # Optional
    #prepareOptions: # Optional
    #runTests: true # Optional
    #credentialsOption: 'serviceEndpoint' # Required when runTests == True# Options: serviceEndpoint, inputs
    #serverEndpoint: # Required when runTests == True && CredsType == ServiceEndpoint
    #username: # Required when runTests == True && CredsType == Inputs
    #password: # Required when runTests == True && CredsType == Inputs
    #appSlug: # Required when runTests == True
    #devices: # Required when runTests == True
    #series: 'master' # Optional
    #dsymDirectory: # Optional
    #localeOption: 'en_US' # Required when runTests == True# Options: da_DK, nl_NL, en_GB, en_US, fr_FR, de_DE, ja_JP, ru_RU, es_MX, es_ES, user
    #userDefinedLocale: # Optional
    #loginOptions: # Optional
    #runOptions: # Optional
    #skipWaitingForResults: # Optional
    #cliFile: # Optional
    #showDebugOutput: # Optional

Arguments

ArgumentDescription
Binary application file path(Required) Relative path from the repo root to the APK or IPA file that you want to test.
Artifacts directory(Required) Where to place the artifacts produced by the prepare step and used by the run step. This directory will be created if it does not exist.
Prepare tests(Optional) Specify whether to prepare tests. The default is true.
Test framework(Required) Options: appium, calabash, espresso, uitest (Xamarin UI Test), xcuitest
Build directory (Appium)(Required) Path to directory with Appium tests.
Project directory (Calabash)(Required) Path to Calabash workspace directory.
Cucumber config file (Calabash)(Optional) Path to Cucumber configuration file, usually cucumber.yml.
Profile to run (Calabash)(Optional) Profile to run. This value must exists in the Cucumber configuration file.
Skip Configuration Check (Calabash)(Optional) Force running without Cucumber profile.
Signing information (Calabash)(Optional) Use Signing Information for signing the test server.
Build directory (Espresso)(Optional) Path to Espresso output directory.
Test APK path (Espresso)(Optional) Path to APK file with Espresso tests. If not set, build-dir is used to discover it. Wildcard is allowed.
Store file (Xamarin UI Test)(Optional) Path to the store file used to sign the app.
Store password (Xamarin UI Test)(Optional) Password of the store file used to sign the app. Use a new variable with its lock enabled on the Variables tab to encrypt this value.
Key alias (Xamarin UI Test)(Optional) Enter the alias that identifies the public/private key pair used in the store file..
Key password (Xamarin UI Test)(Optional) Enter the key password for the alias and store file. Use a new variable with its lock enabled on the Variables tab to encrypt this value.
Test tools directory (Xamarin UI Test)(Optional) Path to directory with Xamarin UI test tools that contains test-cloud.exe.
Signing information (Xamarin UI Test)(Optional) Use Signing Information for signing the test server.
Build directory (Xamarin UI Test)(Optional) Path to the build output directory (usually $(ProjectDir)/Build/Products/Debug-iphoneos).
Build directory (XCUITest)(Required) Path to directory with built test assemblies.
Test IPA path (XCUITest)(Optional) Path to the *.ipa file with the XCUITest tests.
Additional options (for preparing tests)(Optional) Additional arguments passed to the App Center test prepare step.
Run tests(Optional) Specify whether to run the tests. The default is true.
Authentication method(Required) Use App Center service connection or enter credentials to connect to App Center.
App Center service connection(Required) Select the service connection for App Center. Create a new App Center service connection in Azure DevOps project settings.
App Center username (when not using a service connection)(Required) Visit https://appcenter.ms/settings/profile to get your username.
App Center password (when not using a service connection)(Required) Visit https://appcenter.ms/settings/profile to set your password. It can accept a variable defined in build or release pipelines as '$(passwordVariable)'. You may mark variable type as 'secret' to secure it.
App slug(Required) The app slug is in the format of {username}/{app_identifier}. To locate {username} and {app_identifier} for an app, click on its name from https://appcenter.ms/apps, and the resulting URL is in the format of https://appcenter.ms/users/{username}/apps/{app_identifier}.
Devices(Required) String to identify what devices this test will run against. Copy and paste this string when you define a new test run from App Center Test beacon.
Test series(Optional) The series name for organizing test runs (e.g. master, production, beta).
dSYM directory(Optional) Path to iOS symbol files.
System language(Required) Options: da_DK, de_DE, en_GB, en_US, es_ES, es_MX, fr_FR, ja_JP, nl_NL, ru_RU, user. If your language isn't an option, use user/Other and enter its locale below, such as en_US.
Other locale(Optional) Enter any two-letter ISO-639 language code along with any two-letter ISO 3166 country code in the format [language]_[country], such as en_US.
Additional options for login(Optional) Additional arguments passed to the App Center login step.
Additional options for run(Optional) Additional arguments passed to the App Center test run.
Do not wait for test result(Optional) Specify whether to execute tests asynchronously, exiting just after tests are uploaded, without waiting for test results. The default is false.
App Center CLI location(Optional) Path to the App Center CLI on the build or release agent.
Enable debug output(Optional) Add --debug to the App Center CLI for verbose output.
Control options

Example

This example runs Espresso tests on an Android app using the App Center Test task.

steps:
- task: AppCenterTest@1
  displayName: 'Espresso Test - Synchronous'
  inputs:
    appFile: 'Espresso/espresso-app.apk'
    artifactsDirectory: '$(Build.ArtifactStagingDirectory)/AppCenterTest'
    frameworkOption: espresso
    espressoBuildDirectory: Espresso
    serverEndpoint: 'myAppCenterServiceConnection'
    appSlug: 'xplatbg1/EspressoTests'
    devices: a84c93af

Open source

This task is open source on GitHub. Feedback and contributions are welcome.

Q & A