群集集Cluster sets

适用于:Windows Server 2019Applies to: Windows Server 2019

群集集是 Windows Server 2019 发行版中新的云扩展技术,它在单个软件定义的数据中心 (SDDC) 云按数量级提高群集节点计数。Cluster sets is the new cloud scale-out technology in the Windows Server 2019 release that increases cluster node count in a single Software Defined Data Center (SDDC) cloud by orders of magnitude. 群集集是多个故障转移群集的松耦合分组:计算、存储或超聚合。A cluster set is a loosely-coupled grouping of multiple Failover Clusters: compute, storage or hyper-converged. 群集集技术允许跨群集集内的成员群集和统一存储命名空间中的虚拟机流畅性,以支持虚拟机流畅性。Cluster sets technology enables virtual machine fluidity across member clusters within a cluster set and a unified storage namespace across the set in support of virtual machine fluidity.

在保留成员群集上的现有故障转移群集管理体验的同时,群集集实例还在聚合时提供有关生命周期管理的主要用例。While preserving existing Failover Cluster management experiences on member clusters, a cluster set instance additionally offers key use cases around lifecycle management at the aggregate. 此 Windows Server 2019 应用场景评估指南提供了必需的背景信息以及使用 PowerShell 评估群集集技术的分步说明。This Windows Server 2019 Scenario Evaluation Guide provides you the necessary background information along with step-by-step instructions to evaluate cluster sets technology using PowerShell.

技术简介Technology introduction

开发群集集技术是为了满足特定客户在大规模 (SDDC) 云的操作软件定义数据中心。Cluster sets technology is developed to meet specific customer requests operating Software Defined Datacenter (SDDC) clouds at scale. 群集集价值陈述可能会概括如下:Cluster sets value proposition may be summarized as the following:

  • 通过将多个较小的群集合并为单个大型构造,使支持的 SDDC 云规模大大提高,甚至可将软件故障边界合并为单个群集Significantly increase the supported SDDC cloud scale for running highly available virtual machines by combining multiple smaller clusters into a single large fabric, even while keeping the software fault boundary to a single cluster
  • 通过流畅地将虚拟机迁移到跨此大型构造,管理整个故障转移群集生命周期,包括加入和淘汰群集,而不影响租户虚拟机的可用性Manage entire Failover Cluster lifecycle including onboarding and retiring clusters, without impacting tenant virtual machine availability, via fluidly migrating virtual machines across this large fabric
  • 在超聚合 I 中轻松更改计算与存储的比率Easily change the compute-to-storage ratio in your hyper-converged I
  • 在初始虚拟机放置和后续的虚拟机迁移中,在群集上跨群集实现类似于 Azure 的容错域和可用性集Benefit from Azure-like Fault Domains and Availability sets across clusters in initial virtual machine placement and subsequent virtual machine migration
  • 将 CPU 硬件的不同代混合并匹配到同一个群集集构造中,即使每个容错域都是相同的,才能获得最大效率。Mix-and-match different generations of CPU hardware into the same cluster set fabric, even while keeping individual fault domains homogeneous for maximum efficiency. 请注意,每个单独的群集和整个群集集内仍存在相同硬件的建议。Please note that the recommendation of same hardware is still present within each individual cluster as well as the entire cluster set.

从较高的层次来看,这就是群集集的外观。From a high level view, this is what cluster sets can look like.


以下内容提供了上图中每个元素的快速摘要:The following provides a quick summary of each of the elements in the above image:

管理群集Management cluster

群集集中的管理群集是一个故障转移群集,它承载整个群集集的高度可用管理平面和统一存储命名空间, (群集集命名空间) 引用横向扩展文件服务器 (SOFS) 。Management cluster in a cluster set is a Failover Cluster that hosts the highly-available management plane of the entire cluster set and the unified storage namespace (Cluster Set Namespace) referral Scale-Out File Server (SOFS). 管理群集从逻辑上与运行虚拟机工作负荷的成员群集分离。A management cluster is logically decoupled from member clusters that run the virtual machine workloads. 这使得群集集管理平面可以灵活应对任何已本地化的群集范围内的故障,例如,成员群集的功能丢失。This makes the cluster set management plane resilient to any localized cluster-wide failures, e.g. loss of power of a member cluster.

成员群集Member cluster

群集集中的成员群集通常是运行虚拟机和存储空间直通工作负荷的传统超聚合群集。A member cluster in a cluster set is typically a traditional hyper-converged cluster running virtual machine and Storage Spaces Direct workloads. 多个成员群集参与单个群集集部署,形成更大的 SDDC 云结构。Multiple member clusters participate in a single cluster set deployment, forming the larger SDDC cloud fabric. 成员群集在两个主要方面与管理群集不同:成员群集加入容错域和可用性集构造,成员群集的大小也调整为托管虚拟机和存储空间直通工作负荷。Member clusters differ from a management cluster in two key aspects: member clusters participate in fault domain and availability set constructs, and member clusters are also sized to host virtual machine and Storage Spaces Direct workloads. 由于此原因,在群集集中移动到群集边界的群集集虚拟机不能在管理群集上托管。Cluster set virtual machines that move across cluster boundaries in a cluster set must not be hosted on the management cluster for this reason.

群集集命名空间引用 SOFSCluster set namespace referral SOFS

群集集命名空间引用 (群集集命名空间) SOFS 是一个横向扩展文件服务器,其中群集集命名空间 SOFS 上的每个 SMB 共享都是一个 "SimpleReferral 2019" 类型的引用共享。A cluster set namespace referral (Cluster Set Namespace) SOFS is a Scale-Out File Server wherein each SMB Share on the Cluster Set Namespace SOFS is a referral share – of type 'SimpleReferral' newly introduced in Windows Server 2019. 此引用允许服务器消息块 (SMB) 客户端访问成员群集 SOFS 上托管的目标 SMB 共享。This referral allows Server Message Block (SMB) clients access to the target SMB share hosted on the member cluster SOFS. 群集集命名空间引用 SOFS 是一种轻型引用机制,因此不参与 i/o 路径。The cluster set namespace referral SOFS is a light-weight referral mechanism and as such, does not participate in the I/O path. SMB 引用将缓存在每个客户端节点上的永久,群集集命名空间会根据需要动态地自动更新这些引用。The SMB referrals are cached perpetually on the each of the client nodes and the cluster sets namespace dynamically updates automatically these referrals as needed.

群集集主机Cluster set master

在群集集中,成员群集之间的通信松耦合,并通过名为 "群集集主机" (CS-Master) 的新群集资源进行协调。In a cluster set, the communication between the member clusters is loosely coupled, and is coordinated by a new cluster resource called "Cluster Set Master" (CS-Master). 与任何其他群集资源一样,CS-主服务器具有高可用性,并且能够灵活应对各个成员群集故障和/或管理群集节点故障。Like any other cluster resource, CS-Master is highly available and resilient to individual member cluster failures and/or the management cluster node failures. 通过新群集集 WMI 提供程序,CS-主服务器为所有群集集可管理性交互提供管理终结点。Through a new Cluster Set WMI provider, CS-Master provides the management endpoint for all Cluster Set manageability interactions.

群集集辅助角色Cluster set worker

在群集集部署中,CS 主服务器与名为 "群集集辅助角色" 的成员群集上的新群集资源交互, (CS-Worker) 。In a Cluster Set deployment, the CS-Master interacts with a new cluster resource on the member Clusters called "Cluster Set Worker" (CS-Worker). CS-辅助角色充当群集上的唯一联系,以根据 CS 主管的请求协调本地群集交互。CS-Worker acts as the only liaison on the cluster to orchestrate the local cluster interactions as requested by the CS-Master. 此类交互的示例包括虚拟机放置和群集本地资源清点。Examples of such interactions include virtual machine placement and cluster-local resource inventorying. 群集集中的每个成员群集都只有一个 CS 辅助角色实例。There is only one CS-Worker instance for each of the member clusters in a cluster set.

容错域Fault domain

容错域是指当发生故障时,管理员确定的软件和硬件项目的分组可能会故障转移到一起。A fault domain is the grouping of software and hardware artifacts that the administrator determines could fail together when a failure does occur. 虽然管理员可以将一个或多个群集作为容错域一起指定,但每个节点都可以参与可用性集中的容错域。While an administrator could designate one or more clusters together as a fault domain, each node could participate in a fault domain in an availability set. 群集集按设计,将容错域边界确定决定给精通数据中心拓扑的管理员(例如,PDU、网络)。Cluster sets by design leaves the decision of fault domain boundary determination to the administrator who is well-versed with data center topology considerations – e.g. PDU, networking – that member clusters share.

可用性集Availability set

可用性集可帮助管理员跨容错域配置群集工作负载所需的冗余,方法是将它们组织到可用性集中并将工作负荷部署到该可用性集中。An availability set helps the administrator configure desired redundancy of clustered workloads across fault domains, by organizing those into an availability set and deploying workloads into that availability set. 假设你正在部署双层应用程序,我们建议你在可用性集中为每个层配置至少两个虚拟机,这将确保当可用性集中的一个容错域出现故障时,你的应用程序将至少有一个虚拟机位于同一可用性集的不同容错域中托管的每个层中。Let's say if you are deploying a two-tier application, we recommend that you configure at least two virtual machines in an availability set for each tier which will ensure that when one fault domain in that availability set goes down, your application will at least have one virtual machine in each tier hosted on a different fault domain of that same availability set.

为什么使用群集集Why use cluster sets

群集集可提供大规模权益,而无需牺牲复原能力。Cluster sets provides the benefit of scale without sacrificing resiliency.

群集集允许将多个群集聚集在一起,从而创建一个大型构造,而每个群集在恢复时保持独立。Cluster sets allows for clustering multiple clusters together to create a large fabric, while each cluster remains independent for resiliency. 例如,有多个运行虚拟机的4节点 HCI 群集。For example, you have a several 4-node HCI clusters running virtual machines. 每个群集都提供自身所需的复原能力。Each cluster provides the resiliency needed for itself. 如果存储或内存开始填满,则升级是下一步。If the storage or memory starts to fill up, scaling up is your next step. 进行扩展时,有一些选项和注意事项。With scaling up, there are some options and considerations.

  1. 向当前群集添加更多存储。Add more storage to the current cluster. 使用存储空间直通时,这可能很难,因为完全相同的型号/固件驱动器可能不可用。With Storage Spaces Direct, this may be tricky as the exact same model/firmware drives may not be available. 还需要考虑重新生成时间。The consideration of rebuild times also need to be taken into account.
  2. 添加内存。Add more memory. 如果极限计算机可以处理的内存,会怎么样?What if you are maxed out on the memory the machines can handle? 如果所有可用的内存槽全部可用,会怎么样?What if all available memory slots are full?
  3. 向当前群集添加包含驱动器的其他计算节点。Add additional compute nodes with drives into the current cluster. 这会让我们返回到需要考虑的选项1。This takes us back to Option 1 needing to be considered.
  4. 购买全新的群集Purchase a whole new cluster

这就是群集集提供缩放优点的地方。This is where cluster sets provides the benefit of scaling. 如果将群集添加到群集集,则可以利用其他群集上可用的存储或内存,而无需进行任何额外的购买。If I add my clusters into a cluster set, I can take advantage of storage or memory that may be available on another cluster without any additional purchases. 从复原角度来看,将其他节点添加到存储空间直通不会为仲裁提供额外的投票。From a resiliency perspective, adding additional nodes to a Storage Spaces Direct is not going to provide additional votes for quorum. 此处所述,在停止之前,存储空间直通群集可能会丢失2个节点。As mentioned here, a Storage Spaces Direct Cluster can survive the loss of 2 nodes before going down. 如果有4节点 HCI 群集,3个节点会关闭,从而使整个群集停止运行。If you have a 4-node HCI cluster, 3 nodes go down will take the entire cluster down. 如果有一个8节点群集,3个节点会关闭,从而使整个群集停止运行。If you have an 8-node cluster, 3 nodes go down will take the entire cluster down. 对于集中有两个4节点 HCI 群集的群集集,一个 HCI 中的2个节点会关闭,另一个 HCI 中有1个节点出现故障。With Cluster sets that has two 4-node HCI clusters in the set, 2 nodes in one HCI go down and 1 node in the other HCI go down, both clusters remain up. 最好创建一个大型16节点存储空间直通群集,或将其分解为四个4节点群集并使用群集集吗?Is it better to create one large 16-node Storage Spaces Direct cluster or break it down into four 4-node clusters and use cluster sets? 具有群集集的四个4节点群集提供了相同的规模,但更好的复原能力是,多个计算节点可能会关闭 (异常或维护) 和生产。Having four 4-node clusters with cluster sets gives the same scale, but better resiliency in that multiple compute nodes can go down (unexpectedly or for maintenance) and production remains.

部署群集集的注意事项Considerations for deploying cluster sets

考虑是否需要使用群集集时,请考虑以下问题:When considering if cluster sets is something you need to use, consider these questions:

  • 是否需要超出当前的 HCI 计算和存储规模限制?Do you need to go beyond the current HCI compute and storage scale limits?
  • 所有计算和存储都不完全相同吗?Are all compute and storage not identically the same?
  • 是否在群集之间实时迁移虚拟机?Do you live migrate virtual machines between clusters?
  • 是否需要跨多个群集的 Azure 计算机可用性集和容错域?Would you like Azure-like computer availability sets and fault domains across multiple clusters?
  • 是否需要花时间查看所有群集来确定需要将任何新虚拟机放置在何处?Do you need to take the time to look at all your clusters to determine where any new virtual machines need to be placed?

如果答案为 "是",则需要群集集。If your answer is yes, then cluster sets is what you need.

还有其他几项需要考虑的是,较大的 SDDC 可能会改变总体数据中心策略。There are a few other items to consider where a larger SDDC might change your overall data center strategies. SQL Server 是一个很好的示例。SQL Server is a good example. 在群集之间移动 SQL Server 虚拟机是否需要授权 SQL 才能在其他节点上运行?Does moving SQL Server virtual machines between clusters require licensing SQL to run on additional nodes?

横向扩展文件服务器和群集集Scale-out file server and cluster sets

在 Windows Server 2019 中,有一个名为 "基础结构横向扩展文件服务器 (SOFS) 的新的横向扩展文件服务器角色。In Windows Server 2019, there is a new scale-out file server role called Infrastructure Scale-Out File Server (SOFS).

以下注意事项适用于基础结构 SOFS 角色:The following considerations apply to an Infrastructure SOFS role:

  1. 在一个故障转移群集上,最多只能有一个 SOFS 群集角色。There can be at most only one Infrastructure SOFS cluster role on a Failover Cluster. 基础结构 SOFS 角色是通过将 "-基础结构" 交换机参数指定给ClusterScaleOutFileServerRole cmdlet 创建的。Infrastructure SOFS role is created by specifying the "-Infrastructure" switch parameter to the Add-ClusterScaleOutFileServerRole cmdlet. 例如:For example:

    Add-ClusterScaleoutFileServerRole -Name "my_infra_sofs_name" -Infrastructure
  2. 在故障转移过程中创建的每个 CSV 卷都将使用基于 CSV 卷名的自动生成的名称来触发创建 SMB 共享。Each CSV volume created in the failover automatically triggers the creation of an SMB Share with an auto-generated name based on the CSV volume name. 管理员不能直接创建或修改 SOFS 角色下的 SMB 共享,而不是通过 CSV 卷创建/修改操作。An administrator cannot directly create or modify SMB shares under an SOFS role, other than via CSV volume create/modify operations.

  3. 在超聚合配置中,基础结构 SOFS 允许 SMB 客户端 (Hyper-v 主机) 与 SOFS SMB 服务器的 (CA) 通信。In hyper-converged configurations, an Infrastructure SOFS allows an SMB client (Hyper-V host) to communicate with guaranteed Continuous Availability (CA) to the Infrastructure SOFS SMB server. 通过虚拟机访问虚拟磁盘 (VHDx) 文件(在客户端和服务器之间转发拥有的虚拟机标识)来实现此超聚合 SMB 环回 CA。This hyper-converged SMB loopback CA is achieved via virtual machines accessing their virtual disk (VHDx) files where the owning virtual machine identity is forwarded between the client and server. 与之前一样,此标识转发使 ACL 的 VHDx 文件与标准的超聚合群集配置类似。This identity forwarding allows ACL-ing VHDx files just as in standard hyper-converged cluster configurations as before.

创建群集集后,群集集命名空间将依赖于每个成员群集上的基础结构 SOFS,此外还会在管理群集中 SOFS 基础结构。Once a cluster set is created, the cluster set namespace relies on an Infrastructure SOFS on each of the member clusters, and additionally an Infrastructure SOFS in the management cluster.

将成员群集添加到群集集时,管理员在该群集上指定基础结构 SOFS 的名称(如果已存在)。At the time a member cluster is added to a cluster set, the administrator specifies the name of an Infrastructure SOFS on that cluster if one already exists. 如果基础结构 SOFS 不存在,则此操作将创建新成员群集上的新基础结构 SOFS 角色。If the Infrastructure SOFS does not exist, a new Infrastructure SOFS role on the new member cluster is created by this operation. 如果成员群集上已存在基础结构 SOFS 角色,则 Add 操作会根据需要隐式地将其重命名为指定的名称。If an Infrastructure SOFS role already exists on the member cluster, the Add operation implicitly renames it to the specified name as needed. 群集集对成员群集上的任何现有的单一实例 SMB 服务器或非基础结构 SOFS 角色进行未利用。Any existing singleton SMB servers, or non-Infrastructure SOFS roles on the member clusters are left unutilized by the cluster set.

创建群集集时,管理员可以选择使用已有的 AD 计算机对象作为管理群集上的命名空间根目录。At the time the cluster set is created, the administrator has the option to use an already-existing AD computer object as the namespace root on the management cluster. 群集集创建操作在管理群集上创建基础结构 SOFS 群集角色,或重命名现有的基础结构 SOFS 角色,就像前面为成员群集所述。Cluster set creation operations create the Infrastructure SOFS cluster role on the management cluster or renames the existing Infrastructure SOFS role just as previously described for member clusters. 管理群集上的基础结构 SOFS 用作群集集命名空间引用 (群集集命名空间) SOFS。The Infrastructure SOFS on the management cluster is used as the cluster set namespace referral (Cluster Set Namespace) SOFS. 这只意味着群集集命名空间 SOFS 上的每个 SMB 共享都是 "SimpleReferral" 类型的引用共享-在 Windows Server 2019 中是新引入的。It simply means that each SMB Share on the cluster set namespace SOFS is a referral share – of type 'SimpleReferral' - newly introduced in Windows Server 2019. 此引用允许 SMB 客户端访问成员群集 SOFS 上托管的目标 SMB 共享。This referral allows SMB clients access to the target SMB share hosted on the member cluster SOFS. 群集集命名空间引用 SOFS 是一种轻型引用机制,因此不参与 i/o 路径。The cluster set namespace referral SOFS is a light-weight referral mechanism and as such, does not participate in the I/O path. SMB 引用将缓存在每个客户端节点上的永久,群集集命名空间会根据需要动态地自动更新这些引用The SMB referrals are cached perpetually on the each of the client nodes and the cluster sets namespace dynamically updates automatically these referrals as needed

创建群集集Creating a Cluster Set


创建群集集时,建议遵循以下先决条件:When creating a cluster set, you following prerequisites are recommended:

  1. 配置运行 Windows Server 2019 的管理客户端。Configure a management client running Windows Server 2019.
  2. 在此管理服务器上安装故障转移群集工具。Install the Failover Cluster tools on this management server.
  3. 创建群集成员 (在每个群集上至少有两个群集具有两个群集共享卷的群集) Create cluster members (at least two clusters with at least two Cluster Shared Volumes on each cluster)
  4. 创建跨越成员群集 (物理或来宾) 的管理群集。Create a management cluster (physical or guest) that straddles the member clusters. 这种方法可确保群集集管理平面仍可用,但可能会导致成员群集失败。This approach ensures that the Cluster sets management plane continues to be available despite possible member cluster failures.


  1. 根据必备组件中的定义,从三个群集创建新群集集。Create a new cluster set from three clusters as defined in the prerequisites. 下图提供了创建群集的示例。The below chart gives an example of clusters to create. 在此示例中,群集集的名称将为CSMASTERThe name of the cluster set in this example will be CSMASTER.

    群集名称Cluster Name 稍后要使用的基础结构 SOFS 名称Infrastructure SOFS Name to be used later
  2. 创建所有群集后,使用以下命令创建群集集主机:Once all the clusters have been created, use the following command to create the cluster set master:

    New-ClusterSet -Name CSMASTER -NamespaceRoot SOFS-CLUSTERSET -CimSession SET-CLUSTER
  3. 使用下面的命令集向群集集添加群集服务器:Use the command set below to add a Cluster Server to the cluster set:

    Add-ClusterSetMember -ClusterName CLUSTER1 -CimSession CSMASTER -InfraSOFSName SOFS-CLUSTER1
    Add-ClusterSetMember -ClusterName CLUSTER2 -CimSession CSMASTER -InfraSOFSName SOFS-CLUSTER2


    如果使用的是静态 IP 地址方案,则必须在ClusterSet命令中包含 -StaticAddress x . x. x. x. x. x. x. x. x。If you are using a static IP Address scheme, you must include -StaticAddress x.x.x.x on the New-ClusterSet command.

  4. 一旦你创建了群集集的群集成员,你就可以列出节点集及其属性。Once you have created the cluster set out of cluster members, you can list the nodes set and its properties. 枚举群集集中的所有成员群集:To enumerate all the member clusters in the cluster set:

    Get-ClusterSetMember -CimSession CSMASTER
  5. 枚举群集集中的所有成员群集(包括管理群集节点):To enumerate all the member clusters in the cluster set including the management cluster nodes:

    Get-ClusterSet -CimSession CSMASTER | Get-Cluster | Get-ClusterNode
  6. 列出成员群集中的所有节点:To list all the nodes from the member clusters:

    Get-ClusterSetNode -CimSession CSMASTER
  7. 列出整个群集集中的所有资源组:To list all the resource groups across the cluster set:

    Get-ClusterSet -CimSession CSMASTER | Get-Cluster | Get-ClusterGroup
  8. 若要验证群集集创建过程是否创建了一个 SMB 共享 (将其标识为 Volume1,或将 CSV 文件夹标记为,ScopeName 作为基础结构文件服务器名称,并将路径作为每个群集成员 CSV 卷的基础结构 SOFS) :To verify that the cluster set creation process created one SMB share (identified as Volume1, or whatever the CSV folder is labeled with, the ScopeName being the Infrastructure File Server name and the path as both) on the Infrastructure SOFS for each cluster member's CSV volume:

    Get-SmbShare -CimSession CSMASTER
  9. 可以收集群集集调试日志以供查看。Cluster set debug logs can be collected for review. 可以为所有成员和管理群集收集群集集和群集调试日志:Both the cluster set and cluster debug logs can be gathered for all members and the management cluster:

    Get-ClusterSetLog -ClusterSetCimSession CSMASTER -IncludeClusterLog -IncludeManagementClusterLog -DestinationFolderPath <path>
  10. 在所有群集集成员之间配置 Kerberos约束委派Configure Kerberos constrained delegation between all cluster set members.

  11. 在群集集中的每个节点上,将跨群集虚拟机的实时迁移身份验证类型配置为 Kerberos。Configure the cross-cluster virtual machine live migration authentication type to Kerberos on each node in the Cluster Set.

    foreach($h in $hosts){ Set-VMHost -VirtualMachineMigrationAuthenticationType Kerberos -ComputerName $h }
  12. 将管理群集添加到群集集中每个节点上的本地管理员组中。Add the management cluster to the local administrators group on each node in the cluster set.

    foreach($h in $hosts){ Invoke-Command -ComputerName $h -ScriptBlock {Net localgroup administrators /add <management_cluster_name>$} }

创建新虚拟机并将其添加到群集集Creating new virtual machines and adding to cluster sets

创建群集集后,下一步是创建新的虚拟机。After creating the cluster set, the next step is to create new virtual machines. 通常情况下,在创建虚拟机并将其添加到群集时,需要在群集上执行一些检查,以确定最好在哪个群集上运行。Normally, when it is time to create virtual machines and add them to a cluster, you need to do some checks on the clusters to see which it may be best to run on. 这些检查包括:These checks could include:

  • 群集节点上有多少内存可用?How much memory is available on the cluster nodes?
  • 群集节点上有多少可用磁盘空间?How much disk space is available on the cluster nodes?
  • 虚拟机是否需要特定的存储要求 (例如,我想让 SQL Server 的虚拟机中转到运行更快驱动器的群集;或者,我的基础结构虚拟机并不重要,并且可以在) 速度较慢的驱动器上运行。Does the virtual machine require specific storage requirements (i.e. I want my SQL Server virtual machines to go to a cluster running faster drives; or, my infrastructure virtual machine is not as critical and can run on slower drives).

解决此问题后,你需要在群集上创建需要该虚拟机的虚拟机。Once this questions are answered, you create the virtual machine on the cluster you need it to be. 群集集的优点之一是,群集集为你执行这些检查,并将虚拟机放置在最佳节点上。One of the benefits of cluster sets is that cluster sets do those checks for you and place the virtual machine on the most optimal node.

以下命令将标识最佳群集并将虚拟机部署到该群集。The below commands will both identify the optimal cluster and deploy the virtual machine to it. 在下面的示例中,创建了一个新的虚拟机,该虚拟机指定至少有 4 gb 的内存可用于虚拟机,并且需要使用1个虚拟处理器。In the below example, a new virtual machine is created specifying that at least 4 gigabytes of memory is available for the virtual machine and that it will need to utilize 1 virtual processor.

  • 确保为虚拟机提供4gbensure that 4gb is available for the virtual machine
  • 设置用于1的虚拟处理器set the virtual processor used at 1
  • 检查以确保至少有10% 的 CPU 可用于虚拟机check to ensure there is at least 10% CPU available for the virtual machine
# Identify the optimal node to create a new virtual machine
$vpcount = 1
$targetnode = Get-ClusterSetOptimalNodeForVM -CimSession CSMASTER -VMMemory $memoryinMB -VMVirtualCoreCount $vpcount -VMCpuReservation 10
$secure_string_pwd = convertto-securestring "<password>" -asplaintext -force
$cred = new-object -typename System.Management.Automation.PSCredential ("<domain\account>",$secure_string_pwd)

# Deploy the virtual machine on the optimal node
Invoke-Command -ComputerName $targetnode.name -scriptblock { param([String]$storagepath); New-VM CSVM1 -MemoryStartupBytes 3072MB -path $storagepath -NewVHDPath CSVM.vhdx -NewVHDSizeBytes 4194304 } -ArgumentList @("\\SOFS-CLUSTER1\VOLUME1") -Credential $cred | Out-Null

Start-VM CSVM1 -ComputerName $targetnode.name | Out-Null
Get-VM CSVM1 -ComputerName $targetnode.name | fl State, ComputerName

完成后,系统会向你提供有关虚拟机及其放置位置的信息。When it completes, you will be given the information about the virtual machine and where it was placed. 在上面的示例中,它将显示为:In the above example, it would show as:

State         : Running
ComputerName  : 1-S2D2

如果没有足够的内存、CPU 容量或磁盘空间来添加虚拟机,您将收到以下错误:If you were to have not enough memory, CPU capacity, or disk space to add the virtual machine, you will receive the following error:

Get-ClusterSetOptimalNodeForVM : A cluster node is not available for this operation.

创建虚拟机后,该虚拟机将显示在指定的特定节点上的 Hyper-v 管理器中。Once the virtual machine has been created, it will be displayed in Hyper-V manager on the specific node specified. 若要将其添加为群集集虚拟机,并将其添加到群集,请使用以下命令:To add it as a cluster set virtual machine, and add it to the cluster, use the command below:

Register-ClusterSetVM -CimSession CSMASTER -MemberName $targetnode.Member -VMName CSVM1

完成后,输出将为:When it completes, the output will be:

Id  VMName  State  MemberName  PSComputerName
--  ------  -----  ----------  --------------

如果已将群集添加到现有虚拟机,则还需要向群集集注册虚拟机。If you have added a cluster with existing virtual machines, the virtual machines will also need to be registered with Cluster sets. 要一次注册所有这些虚拟机,请使用以下命令:To register all those virtual machines at once, the command to use is:

Get-ClusterSetMember -Name CLUSTER3 -CimSession CSMASTER | Register-ClusterSetVM -RegisterAll -CimSession CSMASTER

但是,该过程尚未完成,因为需要将虚拟机的路径添加到群集集命名空间中。However, the process is not yet complete, as the path to the virtual machine needs to be added to the cluster set namespace.

例如:添加了现有群集,并且它具有驻留在本地群集共享卷 (CSV) 的预配置虚拟机。For example: An existing cluster is added and it has pre-configured virtual machines which reside on the local Cluster Shared Volume (CSV). VHDX 的路径将类似于 "C:\ClusterStorage\Volume1\MYVM\Virtual Hard Disks\MYVM.vhdx"。The path for the VHDX would be something similar to "C:\ClusterStorage\Volume1\MYVM\Virtual Hard Disks\MYVM.vhdx". 需要完成存储迁移,因为 CSV 路径是由本地设计到单个成员群集,因此一旦虚拟机在成员群集中进行实时迁移,就不能对其进行访问。A storage migration would need to be accomplished, as CSV paths are by design local to a single member cluster and will therefore not be accessible to the virtual machine once they are live migrated across member clusters.

在此示例中,使用 ClusterSetMember 将 CLUSTER3 添加到群集集,并将基础结构横向扩展文件服务器为 SOFS-CLUSTER3。In this example, CLUSTER3 was added to the cluster set using Add-ClusterSetMember with the Infrastructure Scale-Out File Server as SOFS-CLUSTER3. 若要移动虚拟机配置和存储,命令为:To move the virtual machine configuration and storage, the command is:

Move-VMStorage -DestinationStoragePath \\SOFS-CLUSTER3\Volume1 -Name MYVM

完成后,你会收到一条警告:Once it completes, you will receive a warning:

WARNING: There were issues updating the virtual machine configuration that may prevent the virtual machine from running. For more information view the report file below.
WARNING: Report file location: C:\Windows\Cluster\Reports\Update-ClusterVirtualMachineConfiguration '' on date at time.htm.

此警告可以忽略,因为警告为 "未检测到虚拟机角色存储配置中的任何更改"。This warning can be ignored as the warning is "No changes in the virtual machine role storage configuration were detected". 警告的原因是实际的物理位置不更改;仅配置路径。The reason for the warning as the actual physical location does not change; only the configuration paths.

有关 VMStorage 的详细信息,请查看此链接For more information on Move-VMStorage, please review this link.

在不同群集集群集之间实时迁移虚拟机与过去不同。Live migrating a virtual machine between different cluster set clusters is not the same as in the past. 在非群集集方案中,步骤如下:In non-cluster set scenarios, the steps would be:

  1. 从群集中删除虚拟机角色。remove the virtual machine role from the Cluster.
  2. 将虚拟机实时迁移到另一个群集的成员节点。live migrate the virtual machine to a member node of a different cluster.
  3. 将虚拟机作为新虚拟机角色添加到群集中。add the virtual machine into the cluster as a new virtual machine role.

对于群集集,无需执行这些步骤,只需执行一个命令。With Cluster sets these steps are not necessary and only one command is needed. 首先,应使用以下命令将所有网络设置为可用于迁移:First, you should set all networks to be available for the migration with the command:

Set-VMHost -UseAnyNetworkForMigration $true

例如,我想要将群集集的虚拟机从 CLUSTER1 移动到 CLUSTER3 上的 CL3。For example, I want to move a Cluster Set virtual machine from CLUSTER1 to NODE2-CL3 on CLUSTER3. 单个命令为:The single command would be:

Move-ClusterSetVM -CimSession CSMASTER -VMName CSVM1 -Node NODE2-CL3

请注意,这不会移动虚拟机存储或配置文件。Please note that this does not move the virtual machine storage or configuration files. 这并不是必需的,因为虚拟机的路径保留为 \ \ SOFS-CLUSTER1\VOLUME1。This is not necessary as the path to the virtual machine remains as \\SOFS-CLUSTER1\VOLUME1. 向群集集注册虚拟机后,该虚拟机将具有基础结构文件服务器共享路径,并且不需要将驱动器和虚拟机与虚拟机位于同一台计算机上。Once a virtual machine has been registered with cluster sets has the Infrastructure File Server share path, the drives and virtual machine do not require being on the same machine as the virtual machine.

创建可用性集容错域Creating Availability sets Fault Domains

如简介中所述,可以在群集集中配置类似 Azure 的容错域和可用性集。As described in the introduction, Azure-like fault domains and availability sets can be configured in a cluster set. 这对于群集之间的初始虚拟机放置和迁移非常有用。This is beneficial for initial virtual machine placements and migrations between clusters.

在下面的示例中,有四个群集参与群集集。In the example below, there are four clusters participating in the cluster set. 在集内,将使用两个分类创建一个逻辑容错域,并使用其他两个分类创建一个容错域。Within the set, a logical fault domain will be created with two of the clusters and a fault domain created with the other two clusters. 这两个容错域将包含可用性集。These two fault domains will comprise the Availabiilty Set.

在下面的示例中,CLUSTER1 和 CLUSTER2 将位于名为FD1的容错域中,而 CLUSTER3 和 CLUSTER4 将位于名为FD2的容错域中。In the example below, CLUSTER1 and CLUSTER2 will be in a fault domain called FD1 while CLUSTER3 and CLUSTER4 will be in a fault domain called FD2. 可用性集将称为CSMASTER ,并由两个容错域组成。The availability set will be called CSMASTER-AS and be comprised of the two fault domains.

若要创建容错域,请执行以下命令:To create the fault domains, the commands are:

New-ClusterSetFaultDomain -Name FD1 -FdType Logical -CimSession CSMASTER -MemberCluster CLUSTER1,CLUSTER2 -Description "This is my first fault domain"

New-ClusterSetFaultDomain -Name FD2 -FdType Logical -CimSession CSMASTER -MemberCluster CLUSTER3,CLUSTER4 -Description "This is my second fault domain"

若要确保已成功创建它们,可以运行 ClusterSetFaultDomain 并显示其输出。To ensure they have been created successfully, Get-ClusterSetFaultDomain can be run with its output shown.

PS C:\> Get-ClusterSetFaultDomain -CimSession CSMASTER -FdName FD1 | fl *

PSShowComputerName    : True
FaultDomainType       : Logical
ClusterName           : {CLUSTER1, CLUSTER2}
Description           : This is my first fault domain
FDName                : FD1
Id                    : 1
PSComputerName        : CSMASTER

现在,已创建了容错域,需要创建可用性集。Now that the fault domains have been created, the availability set needs to be created.

New-ClusterSetAvailabilitySet -Name CSMASTER-AS -FdType Logical -CimSession CSMASTER -ParticipantName FD1,FD2

若要验证是否已创建,请使用:To validate it has been created, then use:

Get-ClusterSetAvailabilitySet -AvailabilitySetName CSMASTER-AS -CimSession CSMASTER

创建新虚拟机时,需要在确定最佳节点时使用-AvailabilitySet 参数。When creating new virtual machines, you would then need to use the -AvailabilitySet parameter as part of determining the optimal node. 这样就会如下所示:So it would then look something like this:

# Identify the optimal node to create a new virtual machine
$vpcount = 1
$av = Get-ClusterSetAvailabilitySet -Name CSMASTER-AS -CimSession CSMASTER
$targetnode = Get-ClusterSetOptimalNodeForVM -CimSession CSMASTER -VMMemory $memoryinMB -VMVirtualCoreCount $vpcount -VMCpuReservation 10 -AvailabilitySet $av
$secure_string_pwd = convertto-securestring "<password>" -asplaintext -force
$cred = new-object -typename System.Management.Automation.PSCredential ("<domain\account>",$secure_string_pwd)

由于各种生命周期,从群集集中删除群集。Removing a cluster from cluster sets due to various life cycles. 有时需要从群集集中删除群集。There are times when a cluster needs to be removed from a cluster set. 作为最佳做法,所有群集集的虚拟机都应移出群集。As a best practice, all cluster set virtual machines should be moved out of the cluster. 这可以通过使用ClusterSetVMVMStorage命令来完成。This can be accomplished using the Move-ClusterSetVM and Move-VMStorage commands.

但是,如果还不移动虚拟机,则群集集将运行一系列操作以向管理员提供直观的结果。However, if the virtual machines will not be moved as well, cluster sets runs a series of actions to provide an intuitive outcome to the administrator. 从集中删除群集后,在要删除的群集上托管的所有剩余群集集虚拟机将只会成为与该群集绑定的高可用虚拟机,前提是这些虚拟机有权访问其存储。When the cluster is removed from the set, all remaining cluster set virtual machines hosted on the cluster being removed will simply become highly available virtual machines bound to that cluster, assuming they have access to their storage. 群集集还将通过以下方式自动更新其清单:Cluster sets will also automatically update its inventory by:

  • 不再跟踪正在删除的群集及其上运行的虚拟机的运行状况No longer tracking the health of the now-removed cluster and the virtual machines running on it
  • 从群集集命名空间中删除以及对在现在删除的群集上托管的共享的所有引用Removes from cluster set namespace and all references to shares hosted on the now-removed cluster

例如,从群集集中删除 CLUSTER1 群集的命令将是:For example, the command to remove the CLUSTER1 cluster from cluster sets would be:

Remove-ClusterSetMember -ClusterName CLUSTER1 -CimSession CSMASTER

常见问题 (FAQ)Frequently asked questions (FAQ)

问题: 在我的群集集中,我是否局限于只使用超聚合群集?Question: In my cluster set, am I limited to only using hyper-converged clusters?
答: 否。Answer: No. 可以将存储空间直通与传统分类混合使用。You can mix Storage Spaces Direct with traditional clusters.

问题: 可以通过 System Center Virtual Machine Manager 来管理群集集吗?Question: Can I manage my Cluster Set via System Center Virtual Machine Manager?
答案: System Center Virtual Machine Manager 当前不支持群集集Answer: System Center Virtual Machine Manager does not currently support Cluster sets

问题: Windows Server 2012 R2 或2016群集是否可以在同一群集集中共存?Question: Can Windows Server 2012 R2 or 2016 clusters co-exist in the same cluster set?
问题: 只需让这些群集加入同一群集集,就可以在 Windows Server 2012 R2 或2016群集之外迁移工作负荷了吗?Question: Can I migrate workloads off Windows Server 2012 R2 or 2016 clusters by simply having those clusters join the same Cluster Set?
答案: 群集集是 Windows Server 2019 中引入的一项新技术,因此在以前的版本中不存在。Answer: Cluster sets is a new technology being introduced in Windows Server 2019, so as such, does not exist in previous releases. 基于 OS 的下层群集无法加入群集集。Down-level OS-based clusters cannot join a cluster set. 但是,群集操作系统滚动升级技术应通过将这些群集升级到 Windows Server 2019,提供要查找的迁移功能。However, Cluster Operating System rolling upgrades technology should provide the migration functionality that you are looking for by upgrading these clusters to Windows Server 2019.

问题: 群集集是否允许我) 单独缩放存储或计算 (?Question: Can Cluster sets allow me to scale storage or compute (alone)?
答案: 是的,只需添加存储空间直通或传统的 Hyper-v 群集。Answer: Yes, by simply adding a Storage Space Direct or traditional Hyper-V cluster. 对于群集集,即使在超聚合群集集中,它也会直接更改计算到存储的比率。With cluster sets, it is a straightforward change of Compute-to-Storage ratio even in a hyper-converged cluster set.

问题: 群集集的管理工具是什么Question: What is the management tooling for cluster sets
答案: 此版本中的 PowerShell 或 WMI。Answer: PowerShell or WMI in this release.

问题: 跨群集实时迁移如何与不同代的处理器配合工作?Question: How will the cross-cluster live migration work with processors of different generations?
答案: 群集集不能解决处理器差异,取代 Hyper-v 当前支持的功能。Answer: Cluster sets does not work around processor differences and supersede what Hyper-V currently supports. 因此,处理器兼容模式必须与快速迁移一起使用。Therefore, processor compatibility mode must be used with quick migrations. 对于群集集,建议使用每个单独分类中的相同处理器硬件,并使用整个群集集在群集之间实时迁移。The recommendation for Cluster sets is to use the same processor hardware within each individual Cluster as well as the entire Cluster Set for live migrations between clusters to occur.

问题: 群集是否可以在群集发生故障时自动进行故障转移?Question: Can my cluster set virtual machines automatically failover on a cluster failure?
答案: 在此版本中,只能在群集之间手动实时迁移群集集的虚拟机;但无法自动故障转移。Answer: In this release, cluster set virtual machines can only be manually live-migrated across clusters; but cannot automatically failover.

问题: 如何确保存储对群集故障有复原能力?Question: How do we ensure storage is resilient to cluster failures?
答案: 跨成员群集使用跨群集存储副本 (SR) 解决方案,以实现群集故障的存储弹性。Answer: Use cross-cluster Storage Replica (SR) solution across member clusters to realize the storage resiliency to cluster failures.

问题: 我使用存储副本 (SR) 跨成员群集进行复制。Question: I use Storage Replica (SR) to replicate across member clusters. 是否在将 SR 故障转移到副本目标存储空间直通群集时,群集集命名空间存储 UNC 路径是否发生更改?Do cluster set namespace storage UNC paths change on SR failover to the replica target Storage Spaces Direct cluster?
答案: 在此版本中,此类群集集命名空间引用更改不会与 SR 故障转移一起发生。Answer: In this release, such a cluster set namespace referral change does not occur with SR failover. 如果此方案对你和你计划使用该方案非常重要,请让 Microsoft 知道这种情况。Please let Microsoft know if this scenario is critical to you and how you plan to use it.

问题: 在灾难恢复情况下,是否可以在容错域之间对虚拟机进行故障转移, (假设整个容错域) 关闭?Question: Is it possible to failover virtual machines across fault domains in a disaster recovery situation (say the entire fault domain went down)?
答案: 不能,请注意,目前尚不支持逻辑容错域中的跨群集故障转移。Answer: No, note that cross-cluster failover within a logical fault domain is not yet supported.

问题: 群集集是否可以跨多个站点中的群集 (或 DNS 域) ?Question: Can my cluster set span clusters in multiple sites (or DNS domains)?
答案: 这是未经测试的方案,不会立即计划生产支持。Answer: This is an untested scenario and not immediately planned for production support. 如果此方案对你和你计划使用该方案非常重要,请让 Microsoft 知道这种情况。Please let Microsoft know if this scenario is critical to you and how you plan to use it.

问题: 群集集是否可与 IPv6 一起使用?Question: Does cluster set work with IPv6?
答案: 群集集与故障转移群集一样,都支持 IPv4 和 IPv6。Answer: Both IPv4 and IPv6 are supported with cluster sets as with Failover Clusters.

问题: 群集集的 Active Directory 林要求是什么Question: What are the Active Directory Forest requirements for cluster sets
答案: 所有成员群集都必须在同一个 AD 林中。Answer: All member clusters must be in the same AD forest.

问题: 有多少群集或节点可以作为单个群集集的一部分?Question: How many clusters or nodes can be part of a single cluster Set?
答案: 在 Windows Server 2019 中,群集集经过测试并支持最多64个群集节点。Answer: In Windows Server 2019, cluster sets been tested and supported up to 64 total cluster nodes. 但是,群集集体系结构可扩展到更大的限制,并且不是对限制的硬编码。However, cluster sets architecture scales to much larger limits and is not something that is hardcoded for a limit. 如果你和你计划使用它的规模是否更大,请让 Microsoft 知道。Please let Microsoft know if larger scale is critical to you and how you plan to use it.

问题: 群集集中的所有存储空间直通群集是否构成单个存储池?Question: Will all Storage Spaces Direct clusters in a cluster set form a single storage pool?
答: 否。Answer: No. 存储空间直通技术仍在单个群集内运行,而不是在群集集中的成员群集上运行。Storage Spaces Direct technology still operates within a single cluster and not across member clusters in a cluster set.

问题: 群集集命名空间是否具有高可用性?Question: Is the cluster set namespace highly available?
答案: 是的,群集集命名空间是通过在管理群集上运行的 (CA) 引用 SOFS 命名空间服务器提供的。Answer: Yes, the cluster set namespace is provided via a Continuously Available (CA) referral SOFS namespace server running on the management cluster. Microsoft 建议从成员群集中有足够数量的虚拟机,使其能够应对本地化的群集范围内的故障。Microsoft recommends having enough number of virtual machines from member clusters to make it resilient to localized cluster-wide failures. 但是,若要考虑不可预见的灾难性故障(例如,管理群集中的所有虚拟机同时关闭),则在每个群集集节点上,甚至在重新启动时也会永久缓存引用信息。However, to account for unforeseen catastrophic failures – e.g. all virtual machines in the management cluster going down at the same time – the referral information is additionally persistently cached in each cluster set node, even across reboots.

问题: 群集集基于命名空间的存储访问是否会减缓群集集中的存储性能?Question: Does the cluster set namespace-based storage access slow down storage performance in a cluster set?
答: 否。Answer: No. 群集集命名空间在群集集中提供覆盖引用命名空间-在概念上类似于分布式文件系统命名空间 (DFSN) 。Cluster set namespace offers an overlay referral namespace within a cluster set – conceptually like Distributed File System Namespaces (DFSN). 与 DFSN 不同,所有群集集命名空间引用元数据都是自动填充的,在所有节点上自动更新,而无需管理员干预,因此在存储访问路径中几乎不会产生性能开销。And unlike DFSN, all cluster set namespace referral metadata is auto-populated and auto-updated on all nodes without any administrator intervention, so there is almost no performance overhead in the storage access path.

问题: 如何备份群集集元数据?Question: How can I backup cluster set metadata?
答案: 本指南与故障转移群集的指南相同。Answer: This guidance is the same as that of Failover Cluster. 系统状态备份也会备份群集状态。The System State Backup will backup the cluster state as well. 通过 Windows Server 备份,你可以只还原节点的群集数据库 (这种情况下永远都不需要这样做,因为有一系列自愈逻辑已经) ,或者执行权威还原,在所有节点上回滚整个群集数据库。Through Windows Server Backup, you can do restores of just a node's cluster database (which should never be needed because of a bunch of self-healing logic we have) or do an authoritative restore to roll back the entire cluster database across all nodes. 对于群集集,Microsoft 建议在成员群集上首先执行此类权威还原,然后在需要时执行管理群集。In the case of cluster sets, Microsoft recommends doing such an authoritative restore first on the member cluster and then the management cluster if needed.