AD mode deployment stopped - missing reverse lookup zone entry for DC
Important
The Microsoft SQL Server 2019 Big Data Clusters add-on will be retired. Support for SQL Server 2019 Big Data Clusters will end on February 28, 2025. All existing users of SQL Server 2019 with Software Assurance will be fully supported on the platform and the software will continue to be maintained through SQL Server cumulative updates until that time. For more information, see the announcement blog post and Big data options on the Microsoft SQL Server platform.
Deployment in Active Directory (AD) mode freezes. Check symptoms to see if cause is the domain controller DNS server is missing reverse lookup zone entry.
Symptom
You started deploying BDC with AD mode however the deployment is stuck and not moving forward.
The following example shows the deployment results in a bash shell.
The privacy statement can be viewed at:
https://go.microsoft.com/fwlink/?LinkId=853010
The license terms for SQL Server Big Data Cluster can be viewed at:
Enterprise: https://go.microsoft.com/fwlink/?linkid=2104292
Standard: https://go.microsoft.com/fwlink/?linkid=2104294
Developer: https://go.microsoft.com/fwlink/?linkid=2104079
Cluster deployment documentation can be viewed at:
https://aka.ms/bdc-deploy
NOTE: Cluster creation can take a significant amount of time depending on
configuration, network speed, and the number of nodes in the cluster.
Starting cluster deployment.
Cluster controller endpoint is available at bdc-control.contoso.com:30080, 193.168.5.14:30080.
Waiting for control plane to be ready after 5 minutes.
Waiting for control plane to be ready after 10 minutes.
Waiting for control plane to be ready after 15 minutes.
Waiting for control plane to be ready after 20 minutes.
Waiting for control plane to be ready after 25 minutes.
Check the current deployed pods.
kubectl get pods -n mssql-cluster
The results below indicate that only pods belonging to the controller have been deployed. The pods for compute, data, or storage are not being created.
NAME READY STATUS RESTARTS AGE
control-rts5t 3/3 Running 0 18m
controldb-0 2/2 Running 0 18m
controlwd-csgst 1/1 Running 0 16m
dns-7kfnz 2/2 Running 0 16m
logsdb-0 1/1 Running 0 16m
logsui-2pc29 1/1 Running 0 16m
metricsdb-0 1/1 Running 0 16m
metricsdc-4rtm4 1/1 Running 0 16m
metricsdc-6lr2t 1/1 Running 0 16m
metricsdc-ftx9m 1/1 Running 0 16m
metricsdc-h59jb 1/1 Running 0 16m
metricsui-lvdpt 1/1 Running 0 16m
mgmtproxy-mkmxp 2/2 Running 0 16m
Inspect the security support container logs. Look for LDAP errors.
Check security-support container
Review the security-support container logs.
The following command collects the security-support logs in a cluster at namespace mssql-cluster.
azdata bdc debug copy-logs -n mssql-cluster -c security-support
Extract the logs and locate \mssql-cluster\control-<identifier>\controller\control-rts5t-controller-stdout.log.
Tip
There are multiple ways to collect the logs. Instead of copying the logs with Azure Data CLI (azdata), you can use a notebook in Azure Data Studio.
In Azure Data Studio, connect to the Kubernetes cluster, and run an appropriate troubleshooting notebook. The following are examples of notebooks.
- TSG027 - Observe cluster deployment
- TSG061 - Get tail of all container logs for pods in BDC namespace
- TSG001 - Run
azdata copy-logs
Inspect the logs
Locate the log. The following example points to a controller deployment log.
<folderOfDebugCopyLog>\debuglogs-mssql-cluster-YYYYMMDD-HHMMSS\<namespace>\control-<identifier>\controller\control-<identifier>-controller-stdout.log"
YYYY-MM-DD HH:MM:SS.ms | ERROR | Failed to create AD user account 'cntrl-controller'. Error code: 53. Message: Failed to create user object: Failed to add object 'CN=cntrl-controller,OU=bdc, DC=CONTOSO, DC=com' to 'CONTOSO.COM': Server is unwilling to perform.
YYYY-MM-DD HH:MM:SS.ms | ERROR | Failed to create AD user account 'ldap-user'. Error code: 53. Message: Failed to create user object: Failed to add object 'CN=ldap-user,OU=bdc, DC=CONTOSO, DC=com' to 'CONTOSO.COM': Server is unwilling to perform.
YYYY-MM-DD HH:MM:SS.ms | ERROR | Failed to create AD user account 'nginx-mgmtproxy'. Error code: 53. Message: Failed to create user object: Failed to add object 'CN=nginx-mgmtproxy,OU=bdc, DC=CONTOSO, DC=com' to 'CONTOSO.COM': Server is unwilling to perform.
Cause
The reverse lookup zone entry for the domain controller in the domain controller DNS entry is missing.
Resolution
Run the following PowerShell script to confirm if you have reverse DNS entry (PTR record) configured.
#Domain Controller FQDN 'DCserver01.contoso.local'
$Domain_controller_FQDN = 'DCserver01.contoso.local'
#Performing Domain Controller DNS record, reverse PTR Checks...
$DcControllerDnsPtr_Result = New-Object System.Collections.ArrayList
try {
$Domain_controller_DNS_Record = Resolve-DnsName $Domain_controller_FQDN -Type A -Server $Domain_DNS_IP_address -ErrorAction Stop
foreach ($ip in $Domain_controller_DNS_Record.IPAddress) {
#resolving hostname by IP address to make sure we have reverse PTR record
if ((Resolve-DnsName $ip).NameHost -eq $Domain_controller_FQDN) {
[void]$DcControllerDnsPtr_Result.add("OK - $Domain_controller_FQDN has an A record with an IP $ip, Reverse PTR record is in place")
}
else {
[void]$DcControllerDnsPtr_Result.add("Missing - $Domain_controller_FQDN has an A record with an IP $ip, But no reverse PTR record was found for the host")
}
}
}
catch {
[void]$DcControllerDnsPtr_Result.add("Error - " + $_.exception.message)
}
#show the results
$DcControllerDnsPtr_Result
Next steps
Verify reverse DNS entry (PTR record) for domain controller.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for