Create S2D Cluster fails with Error 25325 "An error occurred while performing the operation"

Daniel Ionita 96 Reputation points
2021-01-14T17:54:38.133+00:00

Hi guys,

Hoping someone could point me in the right direction here...

Got 4 nodes provisioned in our VMM 2019 (latest build) server. The nodes are Server 2019 Datacenter core up-to-date (Jan '21).

We're trying to use VMM to provision an S2D Cluster. The Job works flawless all the way till Validating the cluster without any warnings. On the "Create Cluster" step it fails and cleans up afterwards.

Permissio-wise:

  • The VMM RunAs is a full Domain Administrator
  • The AD computer objects for the nodes themselves are added to the OUs with full read and Create Computer objects selected
  • The Administrators local security group of each node have as members: the VMM service account AND the VMM computer object

Besides the Validation Report found on each node (which is spotless clean) and the Event Viewer in VMM which basically gives the same cryptic Error, there's nothing to go on here...

Are tehre any other logs I could pursue to trace this down?

Thanks a lot!

System Center Virtual Machine Manager
Windows Server Clustering
Windows Server Clustering
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.Clustering: The grouping of multiple servers in a way that allows them to appear to be a single unit to client computers on a network. Clustering is a means of increasing network capacity, providing live backup in case one of the servers fails, and improving data security.
950 questions
{count} votes

Accepted answer
  1. Daniel Ionita 96 Reputation points
    2021-01-20T09:01:25.017+00:00

    Coming back to this, we ended up creating the cluster manually, then it will appear in VMM to be managed... Worth mentioning that the VMM server is 2019 latest build, but sat on Win Server 2016 Standard. This created problems with RSAT tools being older and misbehaving on some management aspects of the 2019 cluster (for example the Storage Provider was not recognised and kept throwing errors (exact issue here https://social.technet.microsoft.com/Forums/systemcenter/en-US/76fd6d71-49f4-4749-8f13-f2e9e95dcb57/vmm-2019-cannot-add-s2d-storage-provider-windows-server-2019-hyperconverged-cluster?forum=virtualmachinemanager ).

    After in-place upgrading the underlying OS to 2019, removed and re-added storage provider and it all went through.


2 additional answers

Sort by: Most helpful
  1. Daniel Ionita 96 Reputation points
    2021-01-15T09:06:20.373+00:00

    Hi Crystal! Thanks for your answer.

    I have reviewed that link and many more so far without success. There are no Event Logs on any node individually that would suggest a problem during the creation of the cluster.

    I have a feeling this is related directly to the process of creating the cluster remotely failing at the "New-Cluster" cmdlet, but logs are very unhelpful or nonexistent.

    Is there a way to enable additional logging, either on VMM side or on the cluster nodes themselves to help catch the issue?

    Logs I've analyzed so far are:

    • event viewer on VMM server and all nodes
    • C:\Windows\Cluster\Reports on all nodes (only validation reports are here, nothing about cluster creation process)
    • C:\ProgramData\VMMLogs

    The VMM end-script for the Cluster creation reads:

    # Get Host Group 'MARGE'  
    $HostGroup = Get-SCVMHostGroup -ID "c16f9ec8-4390-45ad-88b0-227d4d2f32ad"  
    # Get RunAs Account 'IMPELLING\SCVMMADMIN'  
    $AdminRunAsAccount = Get-SCRunAsAccount -ID "9fac398b-96e6-4ef6-a93f-7ab489f1ff2b"  
      
    # Get Host 'marge04.impelling.gunit, marge03.impelling.gunit, marge02.impelling.gunit, marge01.impelling.gunit'  
    $VMHosts = @()  
    $VMHosts += Get-SCVMHost -ID "ef70b37b-17b6-4219-b0a3-a4b41c2a3807"  
    $VMHosts += Get-SCVMHost -ID "f1b9abb7-27be-4ddb-89f6-c6fd80db3c6a"  
    $VMHosts += Get-SCVMHost -ID "dee7cda7-585b-4d4d-9f25-e8daae6d5f93"  
    $VMHosts += Get-SCVMHost -ID "fb55ecce-6402-4de2-a12b-f66beca60101"  
      
    $StaticIPAddress = @("172.20.20.1")  
      
    $DCBSettings = @()  
    # Get Virtual Network Adapter 'vSMB2, vSMB1, vSMB2, vSMB1, vSMB2, vSMB1, vSMB2, vSMB1'  
    $VirtualNetworkAdapters = @()  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "16f9003a-a8ee-4f88-a247-94cad6ca7e48"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "9fed1603-79ca-4364-b6d1-2485b7785135"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "d860302b-4791-42f4-a73a-b4c093ed98c5"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "89a70237-998f-4b86-89df-014a23b6e6ae"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "777af89b-3fc4-439a-8d80-606d8bb296b3"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "6a6d96bf-2013-43c8-a4dc-bc18481763ef"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "41f5563e-8f9a-4869-8bf5-77e870d90ae1"  
    $VirtualNetworkAdapters += Get-SCVirtualNetworkAdapter -ID "0930de20-ce4b-4f60-be2d-217a93d32cdd"  
      
    $DCBSettingsItem = New-SCDCBSettings -AddDCBSettings -SMBPriority 3 -SMBBandwidth 50 -ClusterHeartbeatPriority 7 -ClusterHeartbeatBandwidth 1 -VirtualNetworkAdapters $VirtualNetworkAdapters  
    $DCBSettings += $DCBSettingsItem  
      
    Install-SCVMHostCluster -ClusterName "MARGE" -JobGroup "ccc29df5-72b9-4387-bf21-8ce0579d6ff0" -RunAsynchronously -EnableS2D -Credential $AdminRunAsAccount -VMHost $VMHosts -ClusterIPAddress $StaticIPAddress -DCBSettings $DCBSettings  
    

    Appreciate the help!

    Thanks!

    0 comments No comments

  2. Daniel Ionita 96 Reputation points
    2021-01-15T10:48:33.69+00:00

    Hi again,

    Coming back with more logs...

    I went as far as enabling an ETL trace in VMM for debugging and identified the segment where it fails:

    [8]1AEC.1E8C::‎2021‎-‎01‎-‎15 09:40:42.238 [Microsoft-VirtualMachineManager-Debug]4,4,private\product\engine\ADHC\Operations\InstallClusterService.cs,98,InstallClusterService.InstallCluster [s#1 58496ms] Failing [ex#13b] New exception = [[(FaultException`1<ErrorInfo>#1abcba3) System.ServiceModel.FaultException`1[Microsoft.VirtualManager.Utils.ErrorInfo]: The creator of this fault did not specify a Reason. (Fault Detail is equal to CreateClusterFailed (25325)).]] (this: (InstallClusterService#[2]) { tbd }, clusterName: (string) "MARGE", nodes: (String[][3]#17c3e99) { "marge04.impelling.gunit", "marge03.impelling.gunit", "marge02.impelling.gunit" }, ignoredNetworks: (String[][2]#217de62) { "172.20.15.0/27", "172.20.17.0/27" }, staticIPs: (String[][0]#3f53e18) { }, addStorage: (Boolean#0) False),{00000000-0000-0000-0000-000000000000},
    [12]0D74.1E38::‎2021‎-‎01‎-‎15 09:40:42.282 [Microsoft-VirtualMachineManager-Debug]4,4,private\product\common\vmmhost\HostHelper\HostHelper.cs,306,Entering [s#27e4a82e] HostHelper`2.CloseProcess(this: (HostDuplexHelper`3<IInstallCluster, InstallClusterService, IClusterProgress>#[5]) {  }),{00000000-0000-0000-0000-000000000000},
    [12]0D74.1E38::‎2021‎-‎01‎-‎15 09:40:42.282 [Microsoft-VirtualMachineManager-Debug]4,4,private\product\common\vmmhost\HostHelper\HostHelper.cs,320,Checking remote interface for communication fault,{00000000-0000-0000-0000-000000000000},
    [12]0D74.1E38::‎2021‎-‎01‎-‎15 09:40:42.282 [Microsoft-VirtualMachineManager-Debug]4,4,private\product\common\vmmhost\HostHelper\HostHelper.cs,367,Process is still running,{00000000-0000-0000-0000-000000000000},
    [12]0D74.1E38::‎2021‎-‎01‎-‎15 09:40:42.282 [Microsoft-VirtualMachineManager-Debug]4,4,private\product\common\vmmhost\HostHelper\HostHelper.cs,374,Requesting process to exit,{00000000-0000-0000-0000-000000000000},
    

    The only odd thing I find in this trace is the fact that staticIPs: (String[][0]#3f53e18) { } is empty, where it should have 172.20.20.1 as the cluster's IP address specified in the wizard (statically not via selecting a drop down IP pool). It ignores the other static subnets fine as it should, but that string is empty for some reason.

    Does someone have any idea if that is by design or it's a genuine abnormality?

    Thanks guys!