App Center Test task

Azure Pipelines | Azure DevOps Server 2020 | Azure DevOps Server 2019 | TFS 2018 | TFS 2017

This task lets you run test suites against an application binary (.apk or .ipa file) using App Center Test.

Note

In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions, runs are called builds, service connections are called service endpoints, stages are called environments, and jobs are called phases.

YAML snippet

# App Center test
# Test app packages with Visual Studio App Center
- task: AppCenterTest@1
  inputs:
    appFile: 
    #artifactsDirectory: '$(Build.ArtifactStagingDirectory)/AppCenterTest' 
    #prepareTests: true # Optional
    #frameworkOption: 'appium' # Required when prepareTests == True# Options: appium, espresso, calabash, uitest, xcuitest
    #appiumBuildDirectory: # Required when prepareTests == True && Framework == Appium
    #espressoBuildDirectory: # Optional
    #espressoTestApkFile: # Optional
    #calabashProjectDirectory: # Required when prepareTests == True && Framework == Calabash
    #calabashConfigFile: # Optional
    #calabashProfile: # Optional
    #calabashSkipConfigCheck: # Optional
    #uiTestBuildDirectory: # Required when prepareTests == True && Framework == Uitest
    #uitestStorePath: # Optional
    #uiTestStorePassword: # Optional
    #uitestKeyAlias: # Optional
    #uiTestKeyPassword: # Optional
    #uiTestToolsDirectory: # Optional
    #signInfo: # Optional
    #xcUITestBuildDirectory: # Optional
    #xcUITestIpaFile: # Optional
    #prepareOptions: # Optional
    #runTests: true # Optional
    #credentialsOption: 'serviceEndpoint' # Required when runTests == True# Options: serviceEndpoint, inputs
    #serverEndpoint: # Required when runTests == True && CredsType == ServiceEndpoint
    #username: # Required when runTests == True && CredsType == Inputs
    #password: # Required when runTests == True && CredsType == Inputs
    #appSlug: # Required when runTests == True
    #devices: # Required when runTests == True
    #series: 'master' # Optional
    #dsymDirectory: # Optional
    #localeOption: 'en_US' # Required when runTests == True# Options: da_DK, nl_NL, en_GB, en_US, fr_FR, de_DE, ja_JP, ru_RU, es_MX, es_ES, user
    #userDefinedLocale: # Optional
    #loginOptions: # Optional
    #runOptions: # Optional
    #skipWaitingForResults: # Optional
    #cliFile: # Optional
    #showDebugOutput: # Optional

Arguments

Argument Description
app
Binary application file path
(Required) Relative path from the repo root to the APK or IPA file that you want to test.
Argument alias: appFile
artifactsDir
Artifacts directory
(Required) Where to place the artifacts produced by the prepare step and used by the run step. This directory will be created if it does not exist.
Default value: $(Build.ArtifactStagingDirectory)/AppCenterTest
Argument alias: artifactsDirectory
enablePrepare
Prepare tests
(Optional) Specify whether to prepare tests.
Default value: true
Argument alias: prepareTests
framework
Test framework
(Required) Options: appium, calabash, espresso, uitest (Xamarin UI Test), xcuitest.
Default value: appium
Argument alias: frameworkOption
appiumBuildDir
Build directory (Appium)
(Required) Path to directory with Appium tests.
Argument alias: appiumBuildDirectory
espressoBuildDir
Build directory (Espresso)
(Optional) Path to Espresso output directory
Argument alias: espressoBuildDirectory
espressoTestApkPath
Test APK path (Espresso)
(Optional) Path to APK file with Espresso tests. If not set, build-dir is used to discover it. Wildcard is allowed.
Argument alias: espressoTestApkFile
calabashProjectDir
Project directory (Calabash)
(Required) Path to Calabash workspace directory.
Argument alias: calabashProjectDirectory
calabashConfigFile
Cucumber config file (Calabash)
(Optional) Path to Cucumber configuration file, usually cucumber.yml.
calabashProfile
Profile to run (Calabash)
(Optional) Profile to run. This value must exists in the Cucumber configuration file.
calabashSkipConfigCheck
Skip Configuration Check (Calabash)
(Optional) Force running without Cucumber profile.
Default value: false
uitestBuildDir
Build directory (Xamarin UI Test)
(Required) Path to directory with built test assemblies
Argument alias: uiTestBuildDirectory
uitestStorePath
Store file (Xamarin UI Test)
(Optional) Path to the store file used to sign the app.
uitestStorePass
Store password (Xamarin UI Test)
(Optional) Password of the store file used to sign the app. Use a new variable with its lock enabled on the Variables tab to encrypt this value.
Argument alias: uiTestStorePassword
uitestKeyAlias
Key alias (Xamarin UI Test)
(Optional) Enter the alias that identifies the public/private key pair used in the store file
uitestKeyPass
Key password (Xamarin UI Test)
(Optional) Enter the key password for the alias and store file. Use a new variable with its lock enabled on the Variables tab to encrypt this value.
Argument alias: uiTestKeyPassword
uitestToolsDir
Test tools directory (Xamarin UI Test)
(Optional) Path to directory with Xamarin UI test tools that contains test-cloud.exe
Argument alias: uiTestToolsDirectory
signInfo
Signing information (Calabash/Xamarin UI Test
(Optional) Use Signing Information for signing the test server.
xcuitestBuildDir
Build directory (XCUITest)
(Optional) Path to the build output directory, usually $(ProjectDir)/Build/Products/Debug-iphoneos).
Argument alias: xcUITestBuildDirectory
xcuitestTestIpaPath
Test IPA path (XCUITest)
(Optional) Path to the *.ipa file with the XCUITest tests.
Argument alias: xcUITestIpaFile
prepareOpts
Additional options (for preparing tests)
(Optional) Additional arguments passed to the App Center test prepare step.
Argument alias: prepareOptions
enableRun
Run tests
(Optional) Specify whether to run the tests.
Default value: true
Argument alias: runTests
credsType
Authentication method
(Required) Use App Center service connection or enter credentials to connect to App Center.
Default value: serviceEndpoint
Argument alias: credentialsOption
serverEndpoint
App Center service connection
(Required) Select the service connection for App Center. Create a new App Center service connection in Azure DevOps project settings.
username
App Center username (when not using a service connection)
(Required) Visit https://appcenter.ms/settings/profile to get your username.
password
App Center password (when not using a service connection)
(Required) Visit https://appcenter.ms/settings/profile to set your password. It can accept a variable defined in build or release pipelines as $(passwordVariable). You may mark variable type as secret to secure it.
appSlug
App slug
(Required) The app slug is in the format of {username}/{app_identifier}. To locate {username} and {app_identifier} for an app, click on its name from https://appcenter.ms/apps, and the resulting URL is in the format of https://appcenter.ms/users/{username}/apps/{app_identifier}.|
devices
Devices
(Required) String to identify what devices this test will run against. Copy and paste this string when you define a new test run from App Center Test beacon.
series
Test series
(Optional) The series name for organizing test runs (e.g. master, production, beta).
Default value: master
dsymDir
dSYM directory
(Optional) Path to iOS symbol files.
Argument alias: dsymDirectory
locale
System language
(Required) Options: da_DK, de_DE, en_GB, en_US, es_ES, es_MX, fr_FR, ja_JP, nl_NL, ru_RU, user. If your language isn't an option, use user/Other and enter its locale below, such as en_US.
Default value: en_US
Argument alias: localeOption
userDefinedLocale
Other locale
(Optional) Enter any two-letter ISO-639 language code along with any two-letter ISO 3166 country code in the format [language]_[country], such as en_US.
loginOpts
Additional options for login
(Optional) Additional arguments passed to the App Center login step.
Argument alias: loginOptions
runOpts
Additional options for run
(Optional) Additional arguments passed to the App Center test run.
Argument alias: runOptions
async
Do not wait for test result
(Optional) Specify whether to execute tests asynchronously, exiting just after tests are uploaded, without waiting for test results.
Default value: false
Argument alias: skipWaitingForResults
cliLocationOverride
App Center CLI location
(Optional) Path to the App Center CLI on the build or release agent.
Argument alias: cliFile
debug
Enable debug output
(Optional) Add --debug to the App Center CLI for verbose output.
Argument alias: showDebugOutput

Example

This example runs Espresso tests on an Android app using the App Center Test task.

steps:
- task: AppCenterTest@1
  displayName: 'Espresso Test - Synchronous'
  inputs:
    appFile: 'Espresso/espresso-app.apk'
    artifactsDirectory: '$(Build.ArtifactStagingDirectory)/AppCenterTest'
    frameworkOption: espresso
    espressoBuildDirectory: Espresso
    serverEndpoint: 'myAppCenterServiceConnection'
    appSlug: 'xplatbg1/EspressoTests'
    devices: a84c93af

Open source

This task is open source on GitHub. Feedback and contributions are welcome.

FAQ