了解存储空间直通中的缓存Understanding the cache in Storage Spaces Direct

适用于:Windows Server 2019、Windows Server 2016Applies to: Windows Server 2019, Windows Server 2016

存储空间直通具有内置服务器端缓存,以最大限度地提高存储性能。Storage Spaces Direct features a built-in server-side cache to maximize storage performance. 这是一个大型、持久且实时的读取写入缓存。It is a large, persistent, real-time read and write cache. 启用存储空间直通时将自动配置缓存。The cache is configured automatically when Storage Spaces Direct is enabled. 在大多数情况下,无需任何手动管理。In most cases, no manual management whatsoever is required. 缓存的工作原理取决于已有的驱动器类型。How the cache works depends on the types of drives present.

以下视频将详细介绍缓存如何用于存储空间直通,及其他设计注意事项。The following video goes into details on how caching works for Storage Spaces Direct, as well as other design considerations.

存储空间直通设计注意事项Storage Spaces Direct design considerations
(20 分钟)(20 minutes)

驱动器类型和部署选项Drive types and deployment options

存储空间直通目前适用于三种类型的存储设备:Storage Spaces Direct currently works with three types of storage devices:

NVMe(高速非易失性内存)NVMe (Non-Volatile Memory Express)
SATA/SAS SSD(固态硬盘)SATA/SAS SSD (Solid-State Drive)
HDD(硬盘驱动器)HDD (Hard Disk Drive)

有六种组合方式,可以归为两个类别:“全闪存”和“混合”。These can be combined in six ways, which we group into two categories: "all-flash" and "hybrid".

全闪存部署可能性All-flash deployment possibilities

全闪存部署旨在最大限度地提高存储性能,并且不包含旋转硬盘驱动器 (HDD)。All-flash deployments aim to maximize storage performance and do not include rotational hard disk drives (HDD).

All-Flash-Deployment-Possibilities

混合部署可能性Hybrid deployment possibilities

混合部署旨在平衡性能和容量需求或最大限度地提高容量,并且不包含旋转硬盘驱动器 (HDD)。Hybrid deployments aim to balance performance and capacity or to maximize capacity and do include rotational hard disk drives (HDD).

Hybrid-Deployment-Possibilities

自动选择缓存驱动器Cache drives are selected automatically

在具有多个驱动器类型的部署中,存储空间直通自动将所有“最快”类型的驱动器用于缓存。In deployments with multiple types of drives, Storage Spaces Direct automatically uses all drives of the "fastest" type for caching. 剩余的驱动器用作容量空间。The remaining drives are used for capacity.

根据以下层次结构确定“最快的”类型。Which type is "fastest" is determined according to the following hierarchy.

Drive-Type-Hierarchy

例如,如果同时有 NVMe 和 SSD,NVMe 将为 SSD 提供缓存。For example, if you have NVMe and SSDs, the NVMe will cache for the SSDs.

如果同时有 SSD 和 HDD,SSD 将为 HDD 提供缓存。If you have SSDs and HDDs, the SSDs will cache for the HDDs.

备注

缓存驱动器不构成可用的存储容量。Cache drives do not contribute usable storage capacity. 存储在缓存中的所有数据也会存储在其他位置,或在取消暂存后立即存储到其他位置。All data stored in the cache is also stored elsewhere, or will be once it de-stages. 这意味着部署的总原始存储容量仅为容量驱动器的总和。This means the total raw storage capacity of your deployment is the sum of your capacity drives only.

当所有驱动器的类型相同时,不会自动配置缓存。When all drives are of the same type, no cache is configured automatically. 你可以选择手动配置耐用性较高的驱动器,使其为相同类型但耐用性较低的驱动器提供缓存,请参阅手动配置一节了解操作方法。You have the option to manually configure higher-endurance drives to cache for lower-endurance drives of the same type – see the Manual configuration section to learn how.

提示

在全 NVMe 或全 SSD 部署中(特别是在规模非常小的部署中),不将任何驱动器“耗费”在缓存上可以极大地提高存储效率。In all-NVMe or all-SSD deployments, especially at very small scale, having no drives "spent" on cache can improve storage efficiency meaningfully.

自动设置缓存行为Cache behavior is set automatically

根据要为其提供缓存的驱动器类型自动确定缓存行为。The behavior of the cache is determined automatically based on the type(s) of drives that are being cached for. 为固态硬盘提供缓存(例如,NVMe 为 SSD 提供缓存)时,只能缓存写入。When caching for solid-state drives (such as NVMe caching for SSDs), only writes are cached. 为硬盘驱动器提供缓存(例如,SSD 为 HDD 提供缓存)时,可以缓存读取和写入。When caching for hard disk drives (such as SSDs caching for HDDs), both reads and writes are cached.

Cache-Read-Write-Behavior

适用于全闪存部署的只写缓存Write-only caching for all-flash deployments

为固态硬盘(NVMe 或 SSD)提供缓存时,只能缓存写入。When caching for solid-state drives (NVMe or SSDs), only writes are cached. 由于许多写入和重新写入可以在缓存中合并,然后仅根据需要取消暂存,因此可以降低容量驱动器的磨损,从而降低容量驱动器的累计流量并延长其生命周期。This reduces wear on the capacity drives because many writes and re-writes can coalesce in the cache and then de-stage only as needed, reducing the cumulative traffic to the capacity drives and extending their lifetime. 因此,我们建议选择耐用性较高、优化写入的驱动器用于缓存。For this reason, we recommend selecting higher-endurance, write-optimized drives for the cache. 容量驱动器可合理地采用较低的写入耐用性。The capacity drives may reasonably have lower write endurance.

由于读取不会显著影响闪存的生命周期,并且固态驱动器通常提供较低的读取延迟,因此不缓存读取:直接从容量驱动器提供读取(除非数据最近写入,尚未取消暂存)。Because reads do not significantly affect the lifespan of flash, and because solid-state drives universally offer low read latency, reads are not cached: they are served directly from the capacity drives (except when the data was written so recently that it has not yet been de-staged). 这允许缓存完全专用于写入,从而最大限度地提高其有效性。This allows the cache to be dedicated entirely to writes, maximizing its effectiveness.

这就导致写入特性(例如,写入延迟)由缓存驱动器规定,而读取特性由容量驱动器规定。This results in write characteristics, such as write latency, being dictated by the cache drives, while read characteristics are dictated by the capacity drives. 两者一致、可预测并且统一。Both are consistent, predictable, and uniform.

混合部署的读取/写入缓存Read/write caching for hybrid deployments

为硬盘驱动器 (HDD) 提供缓存时可以缓存读取写入,为两者提供了类似闪存的低延迟(速度通常高出 ~10 倍)。When caching for hard disk drives (HDDs), both reads and writes are cached, to provide flash-like latency (often ~10x better) for both. 读取缓存存储最近和常用的读取数据以进行快速访问,可最大限度地减少 HDD 的随机流量。The read cache stores recently and frequently read data for fast access and to minimize random traffic to the HDDs. (由于寻找和旋转延迟, 随机访问 HDD 产生的延迟和丢失时间都很重要。)写入缓存可吸收突发, 并与之前一样, 用于合并写入并重新写入数据, 并最大程度地减少到容量驱动器的累计流量。(Because of seek and rotational delays, the latency and lost time incurred by random access to an HDD is significant.) Writes are cached to absorb bursts and, as before, to coalesce writes and re-writes and minimize the cumulative traffic to the capacity drives.

存储空间直通实现了一种算法,即在取消暂存之前取消随机的写入以模拟磁盘的 IO 模式,即使在来自工作负荷(例如虚拟机)的实际 IO 是随机的情况下,这种模式也似乎是序列化的,Storage Spaces Direct implements an algorithm that de-randomizes writes before de-staging them, to emulate an IO pattern to disk that seems sequential even when the actual IO coming from the workload (such as virtual machines) is random. 这可以实现 IOPS 和 HDD 吞吐量的最大化。This maximizes the IOPS and throughput to the HDDs.

在部署中使用所有三种类型的驱动器进行缓存Caching in deployments with drives of all three types

当存在所有三种类型的驱动器时,NVMe 驱动器为 SSD 和 HDD 提供缓存。When drives of all three types are present, the NVMe drives provides caching for both the SSDs and the HDDs. 缓存行为如上所述:只可为 SSD 缓存写入,可为 HDD 缓存读取和写入。The behavior is as described above: only writes are cached for the SSDs, and both reads and writes are cached for the HDDs. 为 HDD 提供缓存的负担均衡分布在缓存驱动器中。The burden of caching for the HDDs is distributed evenly among the cache drives.

总结Summary

此表总结了哪些驱动器用于缓存、哪些驱动器用作容量空间,以及每个部署可能性对应于哪些缓存行为。This table summarizes which drives are used for caching, which are used for capacity, and what the caching behavior is for each deployment possibility.

部署Deployment 缓存驱动器Cache drives 容量驱动器Capacity drives 缓存行为(默认)Cache behavior (default)
所有 NVMeAll NVMe 无(可选:手动配置)None (Optional: configure manually) NVMeNVMe 只写(如果已配置)Write-only (if configured)
所有 SSDAll SSD 无(可选:手动配置)None (Optional: configure manually) SSDSSD 只写(如果已配置)Write-only (if configured)
NVMe + SSDNVMe + SSD NVMeNVMe SSDSSD 只写Write-only
NVMe + HDDNVMe + HDD NVMeNVMe HDDHDD 读取 + 写入Read + Write
SSD + HDDSSD + HDD SSDSSD HDDHDD 读取 + 写入Read + Write
NVMe + SSD + HDDNVMe + SSD + HDD NVMeNVMe SSD + HDDSSD + HDD HDD 为读取 + 写入,SSD 为只写Read + Write for HDD, Write-only for SSD

服务器端体系结构Server-side architecture

在驱动器级别实现缓存:一台服务器中的单独缓存驱动器被绑定至相同服务器中的一个或多个容量驱动器。The cache is implemented at the drive level: individual cache drives within one server are bound to one or many capacity drives within the same server.

由于缓存位于其余 Windows 软件定义的存储堆栈之下,它没有(也不需要)任何概念感知(例如存储空间或容错)。Because the cache is below the rest of the Windows software-defined storage stack, it does not have nor need any awareness of concepts such as Storage Spaces or fault tolerance. 你可以将其视为创建“混合”驱动器(部分闪存、部分磁盘),此驱动器之后将呈现给 Windows。You can think of it as creating "hybrid" (part flash, part disk) drives which are then presented to Windows. 与实际混合驱动器一样,热数据和冷数据在物理媒介的较快部分和较慢部分之间的实时移动对外部来说几乎是不可见的。As with an actual hybrid drive, the real-time movement of hot and cold data between the faster and slower portions of the physical media is nearly invisible to the outside.

鉴于存储空间直通中的复原至少是服务器级别(这意味着数据副本通常写入不同的服务器中;每个服务中最多存在一个副本),缓存中的数据与不在缓存中的数据一样受益于复原。Given that resiliency in Storage Spaces Direct is at least server-level (meaning data copies are always written to different servers; at most one copy per server), data in the cache benefits from the same resiliency as data not in the cache.

Cache-Server-Side-Architecture

例如,使用三向镜像时,任何数据的三个副本都被写入到不同的服务器中,它们将在服务器中进入缓存。For example, when using three-way mirroring, three copies of any data are written to different servers, where they land in cache. 无论它们稍后是否取消暂存,三个副本将始终存在。Regardless of whether they are later de-staged or not, three copies will always exist.

驱动器绑定是动态的Drive bindings are dynamic

缓存驱动器和容量驱动器之间的绑定比率可以是从 1:1 到 1:12 及更高的任意比率。The binding between cache and capacity drives can have any ratio, from 1:1 up to 1:12 and beyond. 无论是添加还是删除驱动器(例如,纵向扩展或失败后),比率都会自动调节。It adjusts dynamically whenever drives are added or removed, such as when scaling up or after failures. 这意味着,你随时可以独立添加缓存驱动器或容量驱动器。This means you can add cache drives or capacity drives independently, whenever you want.

Dynamic-Binding

为了对称,我们建议将容量驱动器的数量设置为缓存驱动器数量的倍数。We recommend making the number of capacity drives a multiple of the number of cache drives, for symmetry. 例如,如果你有 4 个缓存驱动器,则使用 8 个容量驱动器(1:2 的比率),这样一来,你所能体验到的性能将比使用 7 个或 9 个容量驱动器都要高。For example, if you have 4 cache drives, you will experience more even performance with 8 capacity drives (1:2 ratio) than with 7 or 9.

处理缓存驱动器故障Handling cache drive failures

当缓存驱动器发生故障时,本地服务器将丢失任何尚未取消暂存的写入,这意味着它们只存在于其他副本(位于其他服务器)中。When a cache drive fails, any writes which have not yet been de-staged are lost to the local server, meaning they exist only on the other copies (in other servers). 和其他任何驱动器发生故障后一样,存储空间能够通过咨询正常运行的副本自动恢复。Just like after any other drive failure, Storage Spaces can and does automatically recover by consulting the surviving copies.

在短时间内,与丢失缓存的驱动器绑定的容量驱动器将显示不正常。For a brief period, the capacity drives which were bound to the lost cache drive will appear unhealthy. 一旦(自动)发生缓存重绑定且(自动)完成数据修复,它们将恢复显示为正常。Once the cache rebinding has occurred (automatic) and the data repair has completed (automatic), they will resume showing as healthy.

这种情况就是每台服务器需要至少两个缓存驱动器才能保障性能的原因。This scenario is why at minimum two cache drives are required per server to preserve performance.

Handling-Failure

然后,你可以像更换其他任何驱动器一样,更换缓存驱动器。You can then replace the cache drive just like any other drive replacement.

备注

你可能需要关闭电源才能安全更换外接卡 (AIC) 或 M.2 外形规格的 NVMe。You may need to power down to safely replace NVMe that is Add-In Card (AIC) or M.2 form factor.

与其他缓存之间的关系Relationship to other caches

Windows 软件定义存储堆栈中有多个其他无关的缓存。There are several other unrelated caches in the Windows software-defined storage stack. 示例包括存储空间回写缓存和群集共享卷 (CSV) 内存中读取缓存。Examples include the Storage Spaces write-back cache and the Cluster Shared Volume (CSV) in-memory read cache.

如果具有存储空间直通,不应从其默认行为中修改存储空间回写缓存。With Storage Spaces Direct, the Storage Spaces write-back cache should not be modified from its default behavior. 例如,不应使用 New-Volume cmdlet 上的 -WriteCacheSize 等参数。For example, parameters such as -WriteCacheSize on the New-Volume cmdlet should not be used.

你可以选择是否使用 CSV 缓存,由你决定。You may choose to use the CSV cache, or not – it's up to you. 存储空间直通中的 CSV 缓存默认关闭,但它们不会以任何方式与本主题中描述的新缓存冲突。It is off by default in Storage Spaces Direct, but it does not conflict with the new cache described in this topic in any way. 在某些情况下,这可以提供有价值的性能改善。In certain scenarios it can provide valuable performance gains. 有关详细信息,请参阅如何启用 CSV 缓存For more information, see How to Enable CSV Cache.

手动配置Manual configuration

大多数部署无需进行手动配置。For most deployments, manual configuration is not required. 如果需要, 请参阅以下各节。In case you do need it, see the following sections.

如果需要在安装后更改缓存设备模型, 请编辑运行状况服务的支持组件文档, 如运行状况服务概述中所述。If you need to make changes to the cache device model after setup, edit the Health Service's Support Components Document, as described in Health Service overview.

指定缓存驱动器模型Specify cache drive model

在所有驱动器类型相同的部署(例如,全 NVMe 或全 SSD 部署)中不会配置缓存,这是因为在相同类型的驱动器中,Windows 无法自动区分写入耐用性等特性。In deployments where all drives are of the same type, such as all-NVMe or all-SSD deployments, no cache is configured because Windows cannot distinguish characteristics like write endurance automatically among drives of the same type.

若要使用耐用性较高的驱动器为相同类型但耐用性较低的驱动器提供缓存,你可以指定用于 Enable-ClusterS2D cmdlet 的 -CacheDeviceModel 参数的驱动器模型。To use higher-endurance drives to cache for lower-endurance drives of the same type, you can specify which drive model to use with the -CacheDeviceModel parameter of the Enable-ClusterS2D cmdlet. 启用存储空间直通后,这一模型的所有驱动器将全部用于缓存。Once Storage Spaces Direct is enabled, all drives of that model will be used for caching.

提示

请务必与 Get-PhysicalDisk 输出中显示的模型字符串完全匹配。Be sure to match the model string exactly as it appears in the output of Get-PhysicalDisk.

示例Example

首先, 获取物理磁盘列表:First, get a list of physical disks:

Get-PhysicalDisk | Group Model -NoElement

下面是一些示例输出:Here's some example output:

Count Name
----- ----
    8 FABRIKAM NVME-1710
   16 CONTOSO NVME-1520

然后输入以下命令, 并指定缓存设备模型:Then enter the following command, specifying the cache device model:

Enable-ClusterS2D -CacheDeviceModel "FABRIKAM NVME-1710"

通过在 PowerShell 中运行 Get-PhysicalDisk 并验证其 Usage 属性是否显示 "Journal" ,你可以验证预期的驱动器是否用于缓存。You can verify that the drives you intended are being used for caching by running Get-PhysicalDisk in PowerShell and verifying that their Usage property says "Journal".

手动部署可能性Manual deployment possibilities

手动配置可以实现以下部署可能性:Manual configuration enables the following deployment possibilities:

Exotic-Deployment-Possibilities

设置缓存行为Set cache behavior

可以替代默认缓存行为。It is possible to override the default behavior of the cache. 例如,即使在全闪存部署中,你也可以将默认缓存行为设置为缓存读取。For example, you can set it to cache reads even in an all-flash deployment. 除非确定默认缓存行为不适用于你的工作负荷,否则我们不鼓励修改默认行为。We discourage modifying the behavior unless you are certain the default does not suit your workload.

若要重写此行为, 请使用ClusterStorageSpacesDirect cmdlet 及其 -CacheModeSSD-CacheModeHDD参数。To override the behavior, use Set-ClusterStorageSpacesDirect cmdlet and its -CacheModeSSD and -CacheModeHDD parameters. 为固态硬盘提供缓存时,CacheModeSSD 参数设为缓存行为。The CacheModeSSD parameter sets the cache behavior when caching for solid-state drives. 为硬盘驱动器提供缓存时,CacheModeHDD 参数设为缓存行为。The CacheModeHDD parameter sets cache behavior when caching for hard disk drives. 启用存储空间直通后可以随时进行设置。This can be done at any time after Storage Spaces Direct is enabled.

你可以使用ClusterStorageSpacesDirect来验证是否已设置了行为。You can use Get-ClusterStorageSpacesDirect to verify the behavior is set.

示例Example

首先, 获取存储空间直通设置:First, get the Storage Spaces Direct settings:

Get-ClusterStorageSpacesDirect

下面是一些示例输出:Here's some example output:

CacheModeHDD : ReadWrite
CacheModeSSD : WriteOnly

然后, 执行以下操作:Then, do the following:

Set-ClusterStorageSpacesDirect -CacheModeSSD ReadWrite

Get-ClusterS2D

下面是一些示例输出:Here's some example output:

CacheModeHDD : ReadWrite
CacheModeSSD : ReadWrite

调整缓存大小Sizing the cache

应调整缓存大小以适应应用程序和工作负荷的工作集(在任何给定时间主动读取或写入的数据)。The cache should be sized to accommodate the working set (the data being actively read or written at any given time) of your applications and workloads.

这在使用硬盘驱动器的混合部署中尤其重要。This is especially important in hybrid deployments with hard disk drives. 如果活动工作集超过缓存大小,或如果活动工作集过快设置偏离,会增加读取缓存遗漏并需要更主动地取消写入暂存,从而影响总体性能。If the active working set exceeds the size of the cache, or if the active working set drifts too quickly, read cache misses will increase and writes will need to be de-staged more aggressively, hurting overall performance.

你可以使用 Windows 中的内置性能监视器 (PerfMon.exe) 工具检查缓存遗漏率。You can use the built-in Performance Monitor (PerfMon.exe) utility in Windows to inspect the rate of cache misses. 具体来说,你可以将群集存储混合磁盘计数器集中的每秒缓存遗漏读取与部署中的 IOPS 整体读取进行比较。Specifically, you can compare the Cache Miss Reads/sec from the Cluster Storage Hybrid Disk counter set to the overall read IOPS of your deployment. 每个“混合磁盘”对应一个容量驱动器。Each "Hybrid Disk" corresponds to one capacity drive.

例如,2 个缓存驱动器绑定 4 个容量驱动器会导致每台服务器中有 4 个“混合磁盘”对象实例。For example, 2 cache drives bound to 4 capacity drives results in 4 "Hybrid Disk" object instances per server.

Performance-Monitor

虽然没有通用的法则,但是,如果遗漏缓存的读取过多可能会使容量大小不足,你应考虑添加缓存驱动器以扩展缓存。There is no universal rule, but if too many reads are missing the cache, it may be undersized and you should consider adding cache drives to expand your cache. 你随时可以独立添加缓存驱动器或容量驱动器。You can add cache drives or capacity drives independently whenever you want.

请参阅See also