Validate Azure Stack system state
Applies to: Azure Stack integrated systems and Azure Stack Development Kit
As an Azure Stack operator, being able to determine the health and status of your system on-demand is essential. The Azure Stack validation tool (Test-AzureStack) is a PowerShell cmdlet that lets you run a series of tests on your system to identify failures if present. You will typically be asked to run this tool through the privileged end point (PEP) when you contact Microsoft Customer Services Support (CSS) with an issue. With the system-wide health and status information at hand, CSS can collect and analyze detailed logs, focus on the area where the error occurred, and work with you to resolve the issue.
Running the validation tool and accessing results
As stated previously, the validation tool is run via the PEP. Each test returns a PASS/FAIL status in the PowerShell window. Here is an outline of the end-to-end validation testing process:
Access the privileged endpoint (PEP). Run the following commands to establish a PEP session:
Enter-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred
To access the PEP on an ASDK host computer, use AzS-ERCS01 for -ComputerName.
Once you are in the PEP, run:
If any tests report FAIL, run
Get-AzureStackLog. For instructions on an integrated system, see To run Get-AzureStackLog on Azure Stack integrated systems, or on the ASDK, see Run Get-AzureStackLog on an Azure Stack Development Kit (ASDK) system.
The cmdlet gathers logs generated by Test-AzureStack. You should not collect logs or contact CSS if tests report WARN.
If you were instructed to run the validation tool by the CSS, the CSS representative will request the logs you collected to continue troubleshooting your issue.
The validation tool lets you run a series of system-level tests and basic cloud scenarios that provide you an insight to the current state and ascertain issues in your system.
Cloud infrastructure tests
These low impact tests work on an infrastructure level and provide you with information on various system components and functions. Currently, tests are grouped into the following categories:
|Test Category||Argument for -Include and -Ignore|
|Azure Stack ACS Summary||AzsAcsSummary|
|Azure Stack Active Directory Summary||AzsAdSummary|
|Azure Stack Alert Summary||AzsAlertSummary|
|Azure Stack Application Crash Summary||AzsApplicationCrashSummary|
|Azure Stack Backup Share Accessibility Summary||AzsBackupShareAccessibility|
|Azure Stack BMC Summary||AzsStampBMCSummary|
|Azure Stack Cloud Hosting Infrastructure Summary||AzsHostingInfraSummary|
|Azure Stack Cloud Hosting Infrastructure Utilization||AzsHostingInfraUtilization|
|Azure Stack Control Plane Summary||AzsControlPlane|
|Azure Stack Defender Summary||AzsDefenderSummary|
|Azure Stack Hosting Infrastructure Firmware Summary||AzsHostingInfraFWSummary|
|Azure Stack Infrastructure Capacity||AzsInfraCapacity|
|Azure Stack Infrastructure Performance||AzsInfraPerformance|
|Azure Stack Infrastructure Role Summary||AzsInfraRoleSummary|
|Azure Stack Network Infra||AzsNetworkInfra|
|Azure Stack Portal and API Summary||AzsPortalAPISummary|
|Azure Stack Scale Unit VM Events||AzsScaleUnitEvents|
|Azure Stack Scale Unit VM Resources||AzsScaleUnitResources|
|Azure Stack Scenarios||AzsScenarios|
|Azure Stack SDN Validation Summary||AzsSDNValidation|
|Azure Stack Service Fabric Role Summary||AzsSFRoleSummary|
|Azure Stack Storage Data Plane||AzsStorageDataPlane|
|Azure Stack Storage Services Summary||AzsStorageSvcsSummary|
|Azure Stack SQL Store Summary||AzsStoreSummary|
|Azure Stack Update Summary||AzsInfraUpdateSummary|
|Azure Stack VM Placement Summary||AzsVmPlacement|
Cloud scenario tests
In addition to the infrastructure tests above, you also have the ability to run cloud scenario tests to check functionality across infrastructure components. Cloud administrator credentials are required to run these tests as they involve resource deployment.
Currently you cannot run cloud scenario tests using Active Directory Federated Services (AD FS) credentials.
The following cloud scenarios are tested by the validation tool:
- Resource group creation
- Plan creation
- Offer creation
- Storage account creation
- Virtual machine creation
- Blob storage operation
- Queue storage operation
- Table storage operation
The parameter List can be used to display all available test categories.
The parameters Include and Ignore can be used to include or exclude test categories. See the following section for more information about the information to be used with these arguments.
Test-AzureStack -Include AzsSFRoleSummary, AzsInfraCapacity
Test-AzureStack -Ignore AzsInfraPerformance
A tenant VM is deployed as part of one the cloud scenario tests. You can use DoNotDeployTenantVm to disable this.
You need to supply the ServiceAdminCredential parameter to run cloud scenario tests as described in the Use case examples section.
BackupSharePath and BackupShareCredential are used when testing infrastructure backup settings as shown in the Use case examples section.
DetailedResults can be used to get pass/fail/warning information for each test, as well as the overall run. When not specified, Test-AzureStack returns $true if there are no failures, and $false if there are failures.
TimeoutSeconds can be used to set a specific time for each group to complete.
The validation tool also supports common PowerShell parameters: Verbose, Debug, ErrorAction, ErrorVariable, WarningAction, WarningVariable, OutBuffer, PipelineVariable, and OutVariable. For more information, see About Common Parameters.
Use case examples
Run validation without cloud scenarios
Run the validation tool without the ServiceAdminCredential parameter to skip running cloud scenario tests:
New-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack
Run validation with cloud scenarios
Supplying the validation tool with the ServiceAdminCredentials parameter runs the cloud scenario tests by default:
Enter-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack -ServiceAdminCredential "<Cloud administrator user name>"
If you wish to run ONLY cloud scenarios without running the rest of the tests, you can use the Include parameter to do so:
Enter-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack -ServiceAdminCredential "<Cloud administrator user name>" -Include AzsScenarios
The cloud administrator user name must be typed in the UPN format: firstname.lastname@example.org (Azure AD). When prompted, type the password to the cloud administrator account.
To improve the operator experience, a Group parameter has been enabled to run multiple test categories at the same time. Currently, there are 3 groups defined: Default, UpdateReadiness and SecretRotationReadiness.
Default: Considered to be a standard run of Test-AzureStack. This group is run by default if no other groups are selected.
UpdateReadiness: A check to see if the stamp can be updated. When the UpdateReadiness group is run, warnings are displayed as errors in the console output, and should be considered as blockers for the update. The following categories are part of the UpdateReadiness group:
SecretRotationReadiness: A check to see if the stamp is in a in which secret rotation can be run. When the SecretRotationReadiness group is run, warnings are displayed as errors in the console output and should be considered as blockers for secret rotation. The following categories are part of the SecretRotationReadiness Group:
Group parameter example
The following example runs Test-AzureStack to test system readiness before installing an update or hotfix using Group. Before you start the installation of an update or hotfix, you should run Test-AzureStack to check the status of your Azure Stack:
Test-AzureStack -Group UpdateReadiness
However, if your Azure Stack is running a version below 1811, use the following PowerShell commands to run Test-AzureStack:
New-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack -Include AzsControlPlane, AzsDefenderSummary, AzsHostingInfraSummary, AzsHostingInfraUtilization, AzsInfraCapacity, AzsInfraRoleSummary, AzsPortalAPISummary, AzsSFRoleSummary, AzsStampBMCSummary
Run validation tool to test infrastructure backup settings
Before configuring infrastructure backup, you can test the backup share path and credential using the AzsBackupShareAccessibility test:
Enter-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack -Include AzsBackupShareAccessibility -BackupSharePath "\\<fileserver>\<fileshare>" -BackupShareCredential $using:backupcred
After configuring backup, you can run AzsBackupShareAccessibility to validate the share is accessible from the ERCS:
Enter-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack -Include AzsBackupShareAccessibility
To test new credentials with the configured backup share, run:
Enter-PSSession -ComputerName "<ERCS VM-name/IP address>" -ConfigurationName PrivilegedEndpoint -Credential $localcred Test-AzureStack -Include AzsBackupShareAccessibility -BackupShareCredential "<PSCredential for backup share>"
Run validation tool to test network infrastructure
This test checks the connectivity of the network infrastructure bypassing the Azure Stack software defined network (SDN). It demonstrates connectivity from a Public VIP to the configured DNS forwarders, NTP servers, and authentication endpoints. This includes connectivity to Azure when using Azure AD as identity provider or the federated server when using ADFS as identity provider.
Include the debug parameter to get a detailed output of the command:
Test-AzureStack -Include AzsNetworkInfra -Debug
To learn more about Azure Stack diagnostics tools and issue logging, see Azure Stack diagnostics tools.
To learn more about troubleshooting, see Microsoft Azure Stack troubleshooting.