Error on Azure Kubernetes Service on Azure Stack HCI Error Fetching Billing Information

Christopher Low Kin Siong 121 Reputation points Microsoft Vendor
2021-08-10T08:56:37.06+00:00

This is a "brand new" azure stack hci cluster
its 4 node dell emc ax 640

aks version 1.0.1.10628
azure stack hci version 10.0.17784 (should be latest non preview build)

Azure Kubernetes Service on Azure Stack HCI

Fetching billing information for this setup failed. Some dashboard functionalities like 'Add cluster' will be disabled in the meanwhile. Reload page to retry

Message
Getting billing status information failed with the error: Get billing status failed. C:\Program Files\AksHci\kvactl.exe cluster addons billing get-status --kubeconfig C:\ClusterStorage\SGLOCHCIVol01\1.0.1.10628\kubeconfig-mgmt --outputformat=json returned a non zero exit code 1 [Error: Get "https://10.36.146.111:6443/api?timeout=10s": context deadline exceeded (Client.Timeout exceeded while awaiting headers)]

An error occurred while fetching clusters. Error: C:\Program Files\AksHci\kubectl.exe --kubeconfig=C:\ClusterStorage\SGLOCHCIVol01\1.0.1.10628\kubeconfig-mgmt get akshciclusters -o json returned a non zero exit code 1 [Unable to connect to the server: dial tcp 10.36.146.111:6443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.]

---
basically, we did a demo of mars/site recovery. this broke the reporting of the cluster to azure as documented in
https://learn.microsoft.com/en-us/answers/questions/436566/azure-stack-registration-query-failed-service-prob.html

I have run the
[System.Reflection.Assembly]::Load("System.EnterpriseServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a")
$publish = New-Object System.EnterpriseServices.Internal.Publish
$publish.GacInstall("c:\windows\system32\azshci\NewtonSoft.Json.dll")

which i think is what broke my aks.

any clue how to fix it. (or even reverse what I did above. ) (so that I can toggle between having a usable kubernetes and azure hci cluster blade on azure)

Azure Stack HCI
Azure Stack HCI
A hyperconverged infrastructure operating system delivered as an Azure service that provides security, performance, and feature updates.
270 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,867 questions
Accepted answer
  1. Christopher Low Kin Siong 121 Reputation points Microsoft Vendor
    2021-08-11T09:21:08.02+00:00

    I think I should close this and restart. i think the issue whether previously was caused by some form of corruption. the new issue is

    after Uninstall-AksHci runs on all nodes with no issue, I am still unable to create aks service either on gui or powershell.

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. MattMcSpirit-MSFT 561 Reputation points
    2021-08-10T18:59:04.237+00:00

    Hey - can you confirm that the AzSHCI cluster is running normally, and still lists as registered correctly, syncing with Azure etc?

    Have you made any other changes to the AzSHCI Cluster since making the MARS agent change below?

    [System.Reflection.Assembly]::Load("System.EnterpriseServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a")
    $publish = New-Object System.EnterpriseServices.Internal.Publish
    $publish.GacInstall("c:\windows\system32\azshci\NewtonSoft.Json.dll")

    Thanks,
    Matt