了解存储空间直通中的缓存Understanding the cache in Storage Spaces Direct

适用于:Windows Server 2019、Windows Server 2016Applies to: Windows Server 2019, Windows Server 2016

存储空间直通提供一个内置的服务器端缓存用于实现最佳存储性能。Storage Spaces Direct features a built-in server-side cache to maximize storage performance. 此缓存是一个大型的持久性实时读取和写入缓存。It is a large, persistent, real-time read and write cache. 启用存储空间直通时,系统会自动配置该缓存。The cache is configured automatically when Storage Spaces Direct is enabled. 在大多数情况下,无需进行任何手动管理。In most cases, no manual management whatsoever is required. 缓存的工作原理取决于现有的驱动器类型。How the cache works depends on the types of drives present.

以下视频将详细介绍缓存如何用于存储空间直通,及其他设计注意事项。The following video goes into details on how caching works for Storage Spaces Direct, as well as other design considerations.

存储空间直通设计注意事项Storage Spaces Direct design considerations
(20 分钟)(20 minutes)

驱动器类型和部署选项Drive types and deployment options

存储空间直通当前适用于四种类型的存储设备:Storage Spaces Direct currently works with four types of storage devices:

驱动器类型Type of drive 描述Description
PMem PMem 是指永久性内存,一种新的低延迟、高性能存储类型。PMem refers to persistent memory, a new type of low latency, high performance storage.
NVMe NVMe(非易失性快速内存)是指直接位于 PCIe 总线上的固态硬盘。NVMe (Non-Volatile Memory Express) refers to solid-state drives that sit directly on the PCIe bus. 常见外形规格为 2.5 英寸 U.2、PCIe 附加卡 (AIC) 和 M.2。Common form factors are 2.5" U.2, PCIe Add-In-Card (AIC), and M.2. NVMe 提供较高的 IOPS 和 IO 吞吐量,延迟也比目前支持的除 PMem 外的任何其他驱动器类型低。NVMe offers higher IOPS and IO throughput with lower latency than any other type of drive we support today except PMem.
SSD SSD 是指通过传统 SATA 或 SAS 连接的固态硬盘。SSD refers to solid-state drives, which connect via conventional SATA or SAS.
HDD HDD 是指旋转式的磁性硬盘,可提供巨量存储容量。HDD refers to rotational, magnetic hard disk drives, which offer vast storage capacity.

这些类型可按各种方式进行组合,我们将其划分成两大类别:“全闪存”和“混合”。These can be combined in various ways, which we group into two categories: "all-flash" and "hybrid".

全闪存部署的可行性All-flash deployment possibilities

“全部闪存”部署旨在最大化存储性能,不包括机械硬盘 (HDD)。All-flash deployments aim to maximize storage performance and do not include rotational hard disk drives (HDD).

显示所有闪存部署可能性的示意图。

混合部署的可行性Hybrid deployment possibilities

混合部署旨在平衡性能和容量或者最大化容量,包括机械硬盘 (HDD)。Hybrid deployments aim to balance performance and capacity or to maximize capacity and do include rotational hard disk drives (HDD).

显示混合部署可能性的示意图。

自动选择缓存驱动器Cache drives are selected automatically

在使用多种类型的驱动器的部署中,存储空间直通自动将“速度最快”的所有驱动器类型用于缓存。In deployments with multiple types of drives, Storage Spaces Direct automatically uses all drives of the "fastest" type for caching. 剩余的驱动器用于提供容量。The remaining drives are used for capacity.

可根据以下层次结构来确定“最快”的类型。Which type is "fastest" is determined according to the following hierarchy.

图示显示不同驱动器的速度的层次结构,硬盘的速度最慢,NVMe 的速度最快。

例如,如果你有 NVMe 和 SSD,则 NVMe 将为 SSD 提供缓存。For example, if you have NVMe and SSDs, the NVMe will cache for the SSDs.

如果你有 SSD 和 HDD,则 SSD 将为 HDD 提供缓存。If you have SSDs and HDDs, the SSDs will cache for the HDDs.

备注

缓存驱动器不会贡献可用的存储容量。Cache drives do not contribute usable storage capacity. 存储在缓存中的所有数据也会存储在其他位置,或者在解除暂存后存储在其他位置。All data stored in the cache is also stored elsewhere, or will be once it de-stages. 这意味着,部署的原始总存储容量只是容量驱动器的总和。This means the total raw storage capacity of your deployment is the sum of your capacity drives only.

如果所有驱动器属于同一类型,则不会自动配置缓存。When all drives are of the same type, no cache is configured automatically. 可以选择手动将持久性较高的驱动器配置为针对同一类型的持久性较低的驱动器提供缓存 – 请参阅手动配置部分了解操作方法。You have the option to manually configure higher-endurance drives to cache for lower-endurance drives of the same type – see the Manual configuration section to learn how.

提示

在全部使用 NVMe 或全部使用 SSD 的部署中,尤其是在极小规模的部署中,不让任何驱动器“支出”容量用于缓存可在一定程度上改善存储效率。In all-NVMe or all-SSD deployments, especially at very small scale, having no drives "spent" on cache can improve storage efficiency meaningfully.

自动设置缓存行为Cache behavior is set automatically

根据要提供缓存的驱动器类型自动确定缓存的行为。The behavior of the cache is determined automatically based on the type(s) of drives that are being cached for. 为固态硬盘提供缓存时(例如,为 SSD 提供 NVMe 缓存),只缓存写入内容。When caching for solid-state drives (such as NVMe caching for SSDs), only writes are cached. 为机械硬盘提供缓存时(例如 SSD 为 HDD 提供缓存),读取和写入内容都将缓存。When caching for hard disk drives (such as SSDs caching for HDDs), both reads and writes are cached.

显示所有闪存部署与混合部署的缓存、读和写行为的关系图。

全闪存部署的只写缓存Write-only caching for all-flash deployments

为固态硬盘(NVMe 或 SSD)提供缓存时,只缓存写入内容。When caching for solid-state drives (NVMe or SSDs), only writes are cached. 由于许多写入和重写可能在缓存中合并,并只在需要时才解除暂存,因此可以减小容量驱动器的磨损,减少发往容量驱动器的累积流量,并延长驱动器的寿命。This reduces wear on the capacity drives because many writes and re-writes can coalesce in the cache and then de-stage only as needed, reducing the cumulative traffic to the capacity drives and extending their lifetime. 出于此原因,我们建议选择持久性较高且已经过写入优化的驱动器用于缓存。For this reason, we recommend selecting higher-endurance, write-optimized drives for the cache. 容量驱动器可能合理地具有较低的写入持久性。The capacity drives may reasonably have lower write endurance.

由于读取操作不会显著影响闪存的寿命,并且固态硬盘一贯提供较低的读取延迟,因此不会缓存读取数据:读取数据直接从容量驱动器提供(除非数据是最近才写入的,以致尚未解除暂存)。Because reads do not significantly affect the lifespan of flash, and because solid-state drives universally offer low read latency, reads are not cached: they are served directly from the capacity drives (except when the data was written so recently that it has not yet been de-staged). 这样,缓存就可以专门用于写入,并将其有效性最大化。This allows the cache to be dedicated entirely to writes, maximizing its effectiveness.

因此,写入特征(例如写入延迟)由缓存驱动器来支配,而读取特征则是由容量驱动器来支配。This results in write characteristics, such as write latency, being dictated by the cache drives, while read characteristics are dictated by the capacity drives. 两者都是一致、可预测且统一的。Both are consistent, predictable, and uniform.

混合部署的读取/写入缓存Read/write caching for hybrid deployments

为机械硬盘 (HDD) 提供缓存时,会同时缓存读取和写入内容,以便为读取和写入操作提供类似于闪存的延迟(通常可将延迟改善大约 10 倍)。When caching for hard disk drives (HDDs), both reads and writes are cached, to provide flash-like latency (often ~10x better) for both. 读取缓存存储最近和经常读取的数据以进行快速访问,并尽量减少发往 HDD 的随机流量。The read cache stores recently and frequently read data for fast access and to minimize random traffic to the HDDs. (由于 HDD 的寻轨和旋转延迟,随机访问 HDD 时出现的滞后时间和时间损失会很明显。)写入内容将会缓存,以缓解数据突发,并与前面所述一样,写入和重写将会合并,以将发往容量驱动器的累积流量减到最小。(Because of seek and rotational delays, the latency and lost time incurred by random access to an HDD is significant.) Writes are cached to absorb bursts and, as before, to coalesce writes and re-writes and minimize the cumulative traffic to the capacity drives.

存储空间直通实现某种算法来反随机化写入,然后将其解除暂存,这样,即使来自工作负荷(例如虚拟机)的实际 IO 是随机的,也能模拟磁盘的 IO 模式,使其看似是有序的。Storage Spaces Direct implements an algorithm that de-randomizes writes before de-staging them, to emulate an IO pattern to disk that seems sequential even when the actual IO coming from the workload (such as virtual machines) is random. 这可以将 HDD 的 IOPS 和吞吐量最大化。This maximizes the IOPS and throughput to the HDDs.

使用所有上述三种驱动器的部署中的缓存Caching in deployments with drives of all three types

使用所有上述三种类型的驱动器时,NVMe 驱动器将为 SSD 和 HDD 提供缓存。When drives of all three types are present, the NVMe drives provides caching for both the SSDs and the HDDs. 行为如上所述:对于 SSD,仅缓存写入;对于 HDD,同时缓存读取和写入。The behavior is as described above: only writes are cached for the SSDs, and both reads and writes are cached for the HDDs. HDD 的缓存负担均匀分散在各个缓存驱动器之间。The burden of caching for the HDDs is distributed evenly among the cache drives.

摘要Summary

下表汇总了哪些驱动器用于缓存、哪些驱动器用于容量,以及每种部署可行性的缓存行为。This table summarizes which drives are used for caching, which are used for capacity, and what the caching behavior is for each deployment possibility.

部署Deployment 缓存驱动器Cache drives 容量驱动器Capacity drives 缓存行为(默认)Cache behavior (default)
全使用 NVMeAll NVMe 无(可选:手动配置)None (Optional: configure manually) NVMeNVMe 只写(如果已配置)Write-only (if configured)
全使用 SSDAll SSD 无(可选:手动配置)None (Optional: configure manually) SSDSSD 只写(如果已配置)Write-only (if configured)
NVMe + SSDNVMe + SSD NVMeNVMe SSDSSD 只写Write-only
NVMe + HDDNVMe + HDD NVMeNVMe HDDHDD 读取 + 写入Read + Write
SSD + HDDSSD + HDD SSDSSD HDDHDD 读取 + 写入Read + Write
NVMe + SSD + HDDNVMe + SSD + HDD NVMeNVMe SSD + HDDSSD + HDD HDD 为读取 + 写入,SSD 为只写Read + Write for HDD, Write-only for SSD

服务器端体系结构Server-side architecture

缓存在驱动器级别实现:一台服务器中的每个缓存驱动器绑定到同一台服务器中的一个或多个容量驱动器。The cache is implemented at the drive level: individual cache drives within one server are bound to one or many capacity drives within the same server.

由于缓存的级别低于剩余 Windows 软件定义的存储堆栈,因此它不运用,也无需意识到“存储空间”或“容错”之类的概念。Because the cache is below the rest of the Windows software-defined storage stack, it does not have nor need any awareness of concepts such as Storage Spaces or fault tolerance. 可将它看作是创建“混合”(一部分是快闪,一部分是磁盘)驱动器,然后将其提供给 Windows。You can think of it as creating "hybrid" (part flash, part disk) drives which are then presented to Windows. 与实际的混合驱动器一样,在外部几乎看不到物理媒体上速度较快和较慢部分之间冷热数据的实时移动。As with an actual hybrid drive, the real-time movement of hot and cold data between the faster and slower portions of the physical media is nearly invisible to the outside.

假设存储空间直通中的复原功能至少位于服务器级别(这意味着,数据副本始终写入不同的服务器;每个服务器最多有一个副本),则缓存中的数据与不在缓存中的数据享有相同的复原能力。Given that resiliency in Storage Spaces Direct is at least server-level (meaning data copies are always written to different servers; at most one copy per server), data in the cache benefits from the same resiliency as data not in the cache.

显示缓存服务器端体系结构的关系图。

例如,在使用三向镜像时,任何数据的三个副本将写入不同的服务器,然后从服务器进入缓存。For example, when using three-way mirroring, three copies of any data are written to different servers, where they land in cache. 无论以后是否解除暂存这些副本,都始终有三个副本。Regardless of whether they are later de-staged or not, three copies will always exist.

驱动器绑定是动态的Drive bindings are dynamic

缓存驱动器与容量驱动器之间的绑定可以采用任何比率,从 1:1 到 1:12 甚至更高都可以。The binding between cache and capacity drives can have any ratio, from 1:1 up to 1:12 and beyond. 每当添加或移除驱动器(例如扩展时或在故障后)时,这种绑定会动态调整。It adjusts dynamically whenever drives are added or removed, such as when scaling up or after failures. 这意味着,只要有需要,都可以单独添加缓存驱动器或容量驱动器。This means you can add cache drives or capacity drives independently, whenever you want.

显示驱动器绑定的动态方式的动画。

为了对称,容量驱动器的数目最好是缓存驱动器数目的倍数。We recommend making the number of capacity drives a multiple of the number of cache drives, for symmetry. 例如,如果你有 4 个缓存驱动器,则配置 8 个容量驱动器(1:2 比例)的性能比配置 7 个或 9 个容量驱动器更均衡。For example, if you have 4 cache drives, you will experience more even performance with 8 capacity drives (1:2 ratio) than with 7 or 9.

处理缓存驱动器故障Handling cache drive failures

当某个缓存驱动器发生故障时,任何尚未解除暂存的写入内容都不会丢进本地服务器,这意味着,它们只存在于其他副本(其他服务器)中。When a cache drive fails, any writes which have not yet been de-staged are lost to the local server, meaning they exist only on the other copies (in other servers). 就像在其他任何驱动器发生故障后一样,存储空间可以通过查阅幸存的副本来自动恢复。Just like after any other drive failure, Storage Spaces can and does automatically recover by consulting the surviving copies.

有一小段时间,绑定到丢失的缓存驱动器的容量驱动器会显示为不正常。For a brief period, the capacity drives which were bound to the lost cache drive will appear unhealthy. 发生缓存重新绑定(自动)并完成数据修复(自动)之后,这些驱动器将继续显示为正常。Once the cache rebinding has occurred (automatic) and the data repair has completed (automatic), they will resume showing as healthy.

因为会发生这种情况,每台服务器至少需要两个缓存驱动器才能保留良好的性能。This scenario is why at minimum two cache drives are required per server to preserve performance.

动画显示每台服务器至少需要两个缓存驱动器才能保留性能。

然后,可以像更换其他任何驱动器一样更换缓存驱动器。You can then replace the cache drive just like any other drive replacement.

备注

可能需要关闭电源来安全更换外形规格为附加卡 (AIC) 或 M.2 的 NVMe。You may need to power down to safely replace NVMe that is Add-In Card (AIC) or M.2 form factor.

与其他缓存之间的关系Relationship to other caches

Windows 软件定义的存储堆栈中有多个其他不相关的缓存。There are several other unrelated caches in the Windows software-defined storage stack. 例如,存储空间写回缓存,以及群集共享卷 (CSV) 内存中读取缓存。Examples include the Storage Spaces write-back cache and the Cluster Shared Volume (CSV) in-memory read cache.

使用存储空间直通时,不应修改存储空间写回缓存的默认行为。With Storage Spaces Direct, the Storage Spaces write-back cache should not be modified from its default behavior. 例如,不应在 New-Volume cmdlet 中使用 -WriteCacheSize 之类的参数。For example, parameters such as -WriteCacheSize on the New-Volume cmdlet should not be used.

你可以自行决定是否使用 CSV 缓存。You may choose to use the CSV cache, or not – it's up to you. 它不会以任何方式与本主题中所述的缓存发生冲突。It does not conflict with the cache described in this topic in any way. 在某些情况下,它可以提供宝贵的性能增益。In certain scenarios it can provide valuable performance gains. 有关详细信息,请参阅如何启用 CSV 缓存For more information, see How to Enable CSV Cache.

手动配置Manual configuration

大多数部署无需进行手动配置。For most deployments, manual configuration is not required. 如果需要手动配置,请参阅以下部分。In case you do need it, see the following sections.

如果需要在设置后对缓存设备型号进行更改,请根据运行状况服务概述中所述,编辑运行状况服务的支持组件文档。If you need to make changes to the cache device model after setup, edit the Health Service's Support Components Document, as described in Health Service overview.

指定缓存驱动器型号Specify cache drive model

在所有驱动器都属于同一类型的部署中(例如,全使用 NVMe 或全使用 SSD 的部署),由于 Windows 无法自动分辨同一类型的驱动器的特征(例如写入持久性),因此不会配置缓存。In deployments where all drives are of the same type, such as all-NVMe or all-SSD deployments, no cache is configured because Windows cannot distinguish characteristics like write endurance automatically among drives of the same type.

若要使用持久性较高的驱动器为类型相同但持久性较低的驱动器提供缓存,可以使用 Enable-ClusterS2D cmdlet 的 -CacheDeviceModel 参数指定要使用的驱动器型号。To use higher-endurance drives to cache for lower-endurance drives of the same type, you can specify which drive model to use with the -CacheDeviceModel parameter of the Enable-ClusterS2D cmdlet. 启用存储空间直通后,该型号的所有驱动器将用于缓存。Once Storage Spaces Direct is enabled, all drives of that model will be used for caching.

提示

请确保型号字符串与 Get-PhysicalDisk 输出中显示的字符串完全一样。Be sure to match the model string exactly as it appears in the output of Get-PhysicalDisk.

示例Example

首先获取物理磁盘的列表:First, get a list of physical disks:

Get-PhysicalDisk | Group Model -NoElement

下面是一些示例输出:Here's some example output:

Count Name
----- ----
    8 FABRIKAM NVME-1710
   16 CONTOSO NVME-1520

然后输入以下命令并指定缓存设备型号:Then enter the following command, specifying the cache device model:

Enable-ClusterS2D -CacheDeviceModel "FABRIKAM NVME-1710"

可以验证所需的驱动器是否正用于缓存,方法是在 PowerShell 中运行 Get-PhysicalDisk,并确认其 Usage 属性是否显示 "Journal"You can verify that the drives you intended are being used for caching by running Get-PhysicalDisk in PowerShell and verifying that their Usage property says "Journal".

手动部署的可行性Manual deployment possibilities

手动配置支持以下部署可行性:Manual configuration enables the following deployment possibilities:

显示手动部署可能性的关系图。

设置缓存行为Set cache behavior

可以重写缓存的默认行为。It is possible to override the default behavior of the cache. 例如,即使在全闪存部署中,也可将其设置为缓存读取。For example, you can set it to cache reads even in an all-flash deployment. 除非你确定默认行为不符合工作负荷,否则我们不建议修改行为。We discourage modifying the behavior unless you are certain the default does not suit your workload.

若要重写行为,请结合 -CacheModeSSD-CacheModeHDD 参数使用 Set-ClusterStorageSpacesDirect cmdlet。To override the behavior, use Set-ClusterStorageSpacesDirect cmdlet and its -CacheModeSSD and -CacheModeHDD parameters. CacheModeSSD 参数设置在为固态硬盘提供缓存时的缓存行为。The CacheModeSSD parameter sets the cache behavior when caching for solid-state drives. CacheModeHDD 参数设置在为机械硬盘提供缓存时的缓存行为。The CacheModeHDD parameter sets cache behavior when caching for hard disk drives. 可以在启用存储空间直通后的任何时间执行此操作。This can be done at any time after Storage Spaces Direct is enabled.

可以使用 Get-ClusterStorageSpacesDirect 来验证是否已设置该行为。You can use Get-ClusterStorageSpacesDirect to verify the behavior is set.

示例Example

首先获取存储空间直通设置:First, get the Storage Spaces Direct settings:

Get-ClusterStorageSpacesDirect

下面是一些示例输出:Here's some example output:

CacheModeHDD : ReadWrite
CacheModeSSD : WriteOnly

然后执行以下操作:Then, do the following:

Set-ClusterStorageSpacesDirect -CacheModeSSD ReadWrite

Get-ClusterS2D

下面是一些示例输出:Here's some example output:

CacheModeHDD : ReadWrite
CacheModeSSD : ReadWrite

调整缓存大小Sizing the cache

应该根据应用程序和工作负荷的工作集(在任何给定时间主动读取或写入的数据)来调整缓存大小。The cache should be sized to accommodate the working set (the data being actively read or written at any given time) of your applications and workloads.

在使用机械硬盘的混合部署中,调整缓存大小特别重要。This is especially important in hybrid deployments with hard disk drives. 如果活动工作集超过缓存大小,或者活动工作集改变速度太快,则读取缓存未命中次数将会增加,并且需要更频繁地解除暂存写入内容,因而会损害整体性能。If the active working set exceeds the size of the cache, or if the active working set drifts too quickly, read cache misses will increase and writes will need to be de-staged more aggressively, hurting overall performance.

可以使用 Windows 的内置性能监视器 (PerfMon.exe) 实用工具来检查缓存未命中率。You can use the built-in Performance Monitor (PerfMon.exe) utility in Windows to inspect the rate of cache misses. 具体而言,可将“群集存储混合磁盘”计数器集中的“缓存未命中读取次数/秒”与部署的整体读取 IOPS 进行比较。 Specifically, you can compare the Cache Miss Reads/sec from the Cluster Storage Hybrid Disk counter set to the overall read IOPS of your deployment. 每个“混合磁盘”对应于一个容量驱动器。Each "Hybrid Disk" corresponds to one capacity drive.

例如,绑定到 4 个容量驱动器的 2 个缓存驱动器在每台服务器上产生 4 个“混合磁盘”对象实例。For example, 2 cache drives bound to 4 capacity drives results in 4 "Hybrid Disk" object instances per server.

Performance-Monitor

不存在通用的法则,但如果过多的读取未命中缓存,则可能表示缓存的大小不足,你应该考虑添加缓存驱动器来扩展缓存。There is no universal rule, but if too many reads are missing the cache, it may be undersized and you should consider adding cache drives to expand your cache. 随时都可以单独添加缓存驱动器或容量驱动器。You can add cache drives or capacity drives independently whenever you want.

其他参考Additional References