Why R2? Your Next SAN with SMB 3.02 Scale-out File Servers
This month, my team of fellow IT Pro Technical Evangelists are publishing a series of articles on Why Windows Server 2012 R2? that highlights the key enhancements and new capabilities that we're seeing driving particular interest in this latest release of our Windows Server operating system in the field. This article has been updated now that Window Server 2012 R2 is generally available. After reading this article, be sure to catch the full series at:
Controlling Storage Costs without Compromises
STORAGE!** For most IT Pros that I speak with, storage is a huge chunk of their IT budget – often accounting for as much as 30%-40% (more in some cases) of their annual capital IT costs. In Windows Server 2012 R2, we’ve made significant storage enhancements to offer enterprise storage capabilities using commodity hardware to help organizations better optimize their storage costs.
Of course, we also continue to support traditional SAN and NAS technologies among the available storage options with Windows Server 2012 R2, adding cool new features like Offloaded Data Transfers (ODX) for even faster performance with intelligent storage arrays. However, this article discusses the commodity storage solutions that Windows Server 2012 R2 supports as alternatives to these traditional enterprise storage architectures, which can provide comparable or better performance at a reduced cost.
In a prior article, I discussed Automated Tiered Storage with Windows Server 2012 R2 Storage Spaces, our built-in storage virtualization solution. In this article, we’ll discuss “Your Next SAN” – a configuration leveraging commodity hardware, Storage Spaces and SMB 3.02 Scale-out File Servers (SoFS) in Windows Server 2012 R2. This configuration can make virtualized storage accessible to multiple Hyper-V hosts over standard high-speed network connections, on a much more affordable basis than a traditional SAN appliance solution delivers. With the tuning and load-balancing improvements made in Windows Server 2012 R2, we’re seeing performance results that are not only comparable to traditional SANs, but can now also achieve increased performance over traditional SANs, particularly when leveraging higher-speed 40Gbps Ethernet and 56Gbps Infiniband network interfaces.
- Did you miss it? Step-by-Step: Automated Tiered Storage with Windows Server 2012 R2
- Learn more! Get our FREE EBOOK, Introducing Windows Server 2012 R2
- Learn more! What’s new in Windows Server 2012 R2 File Server and SMB 3.02
As we progress through this article, I’ll also provide links to key step-by-step resources that you can leverage to build out “Your Next SAN” in your lab environment for further evaluation.
Enterprise Storage, not just for “Enterprise” anymore!
By making enterprise-class storage solutions more affordable, I’m also finding that smaller organizations, those that previously wouldn’t have had budget for a traditional SAN, are now finding it possible to gain the benefits of enterprise storage virtualization with Windows Server 2012 R2. In fact, if you’re working with smaller organizations, you should also check out the new Cluster-in-a-Box (CiB) OEM hardware solutions. These CiB solutions combine commodity disk hardware with 2-to-4 blade servers in a configuration experience that can be completely setup … in as little as 30-minutes!
- Learn more! Set up a 4-node Cluster-in-a-Box in 30-minutes!
What is a SAN, really?
When you look beneath the covers of modern SAN and NAS storage appliances, we can easily identify three major component categories of which these solutions consist: enterprise-grade Physical Disks connected on a high-speed backplane & interconnect; redundant Storage Controllers that virtualize the physical storage, provide advanced caching algorithms and present the storage to hosts; and, Connectivity Adapters, such as iSCSI and Fibre Channel HBAs, that provide high-speed, multi-path connections to the storage network.
When using Windows Server 2012 R2 as your SAN-like alternative, a very similar component architecture can be used – providing an enterprise-grade storage alternative using commodity hardware for reduced cost. Let’s take a look at each layer of this architecture in more detail, as it pertains to Windows Server 2012 R2 …
Physical Disks and JBOD Enclosures
Windows Server 2012 R2 can leverage the same enterprise-grade SAS SSD and HDD disks that are commonly used in expensive SAN and NAS appliances – generally running at SAS interface speeds up to 6Gb/s today. To make these commodity disks directly accessible to multiple Windows Server 2012 R2 “storage controller” nodes in a SoFS cluster, these physical disks would be placed in one or more inexpensive SAS JBOD (Just a Bunch Of Disks) enclosures.
JBOD enclosures provide a SAS backplane that is used to physically connect these disks to multiple servers via standard high-speed SAS host bus adapters (HBAs) in each server.
Note: In our “Your Next SAN” configuration, simple SAS HBA’s are preferred in each server for connecting to SAS JBOD enclosures, not more expensive SAS RAID controllers. If you already have purchased SAS RAID controllers for your servers, we recommend configuring the RAID controllers for “pass-through” or “non-RAID” mode so that they act like an inexpensive SAS HBA.
To make it easy to select a SAS JBOD enclosure that will provide the necessary hardware features and performance, we have a hardware certification program that identifies Windows Server 2012-certified SAS JBOD enclosures. You can find more details on certified SAS JBOD enclosures at:
- Learn more! Storage Spaces, JBODs and Failover Clustering – A Recipe for Cost-Effective, Highly Available Storage
Storage Controllers with SMB 3.0
When you peel back the covers of a traditional SAN, you’ll find that the Storage Controllers (sometimes also called “Storage Processors” or “Array Controllers”) are usually just a couple of blade servers running either Windows or Linux operating systems with additional software pre-installed by a SAN storage vendor to virtualize, cache, present and manage the physical storage as pooled storage resources. Often times, this additional storage software is represented by SAN vendors as their “secret sauce” that differentiates their solution from commodity storage.
In our “Your Next SAN” configuration, we leverage standard blade or rack-mount server hardware running Windows Server 2012 R2 as the “Storage Controllers” with Scale-out File Servers (SoFS) roles using the latest version of the SMB 3.0 protocol, SMB 3.02.
In fact, SMB 3.0 has been heralded even by third-party storage industry veterans as being the “Future of Storage Protocols” for network environments running Windows Server and Windows client workloads. Read what EMC says about SMB 3.0 in their latest whitepaper:
- Learn more! EMC VNX and VNXe with Microsoft SMB 3.0
The following components in Windows Server 2012 R2 are included as our “secret sauce” for “Storage Controller” software:
- Storage Virtualization: Windows Server 2012 includes Storage Spaces to pool and virtualize physical disks.
Learn more! Build a Storage Spaces Lab with Windows Server 2012 R2
- Storage Caching: Read and write caching is provided at both a Storage Virtualization layer, via commodity SSDs within a Storage Spaces pool, as well as at a Storage Presentation layer, via Cluster-Shared Volume (CSV) caching.
Learn more! Configuring and Optimizing the CSV Cache
- Storage Presentation: Virtualized storage can be presented via SMB 3.02 Scale-out File Servers (SoFS) to Hyper-V hosts and/or SQL Server database servers. Additionally, Windows Server 2012 R2 also includes the ability to present virtualized storage via iSCSI and NFS protocols to other virtualization hosts, such as VMware vSphere and Citrix XenServer.
Learn more! Step-by-Step with SMB 3.02 Scale-Out File Servers
Learn more! Automatic SMB Scale-out Rebalancing in Windows Server 2012 R2
Learn more! Reducing VMware Storage Costs with Windows Server 2012 R2 Storage Spaces
- Storage Management: Windows Server 2012 R2 includes Server Manager as a GUI tool for easily managing virtualized storage, as well as full scripting support via the new PowerShell Storage Spaces module. In addition, for enterprise storage management of “Your Next SAN” side-by-side with your existing storage investments, System Center 2012 R2 Virtual Machine Manager (VMM) includes the ability to manage Storage Spaces, Scale-out File Servers, and SMI-S/SMP compliant third-party storage solutions as a unified “Storage Fabric” from one management console.
Learn more! What’s New in System Center 2012 R2 VMM
High-Speed Connectivity over Commodity Storage Networks
In our “Your Next SAN” configuration, instead of leveraging costly iSCSI HBAs or Fibre Channel HBAs to connect “Storage Controllers” to the storage network, we instead leverage standard Ethernet or Infiniband NICs to connect our Windows Server 2012 R2 SoFS nodes on the storage network. As per-port costs for high-speed 10GbE, 40GbE and 56Gbps Infiniband have dropped over the past few years, these connection choices now present very attractive alternatives to traditional Fibre Channel storage network infrastructures. This is supported by the 2012 research report by the Storage Networking Industry Association (SNIA), 10GbE comes of Age.
- Learn more! 10GbE Comes of Age – July 2012 – SNIA
To support these high-speed network interfaces as efficient storage “Connectivity Adapters”, SMB 3.0 includes two key features “in-the-box” with Windows Server 2012 R2:
- SMB Multichannel – Permitting multiple SMB sessions to be established simultaneously over single and/or multiple high-speed network connections to maximize the utilization of these network storage paths. SMB Multichannel automatically detects when multiple sessions can be managed over 1GbE and faster network connections and enables this feature to fully utilize these paths.
Learn more! The basics of SMB Multichannel in SMB 3.0
- SMB Direct – Automatically detects high-speed network adapters, generally 10GbE and faster, that include hardware support for Remote Direct Memory Access (RDMA) to blast large amounts of data over the wire with very low CPU utilization on sending and receiving servers. At one time, RDMA was leveraged only for expensive High Performance Computing (HPC) clusters found in the research labs of large universities and organizations, but SMB Direct brings RDMA to mainstream storage architectures that can be leveraged in any business organization.
Learn more! What is Remote Direct Memory Access?
When architecting the network connectivity layer of “Your Next SAN”, SMB 3.0 provides a great deal of flexibility to leverage multiple physical network paths as dedicated storage paths, or be consolidated with other network IO workloads on a smaller number of high-speed NICs.
All this, and Performance too!
Leveraging Windows Server 2012 as the software foundation for “Your Next SAN” can certainly help to optimize storage costs, and place you in direct control of independently scaling each layer of your storage investment, based on where it makes the most sense for your IO workloads. But, it also delivers great performance too!
Since the release of SMB 3.0, many of our storage partners have performed benchmark tests to demonstrate the enterprise-grade storage throughput that can be achieved with the configuration outlined in this article. For instance, in the performance tests that Mellanox conducted with their quad data rate (QDR) and fourteen data rate (FDR) RDMA-enabled network adapters, they found that they were able to achieve throughput that was operating at nearly native disk access speeds with SMB 3.0!
- Learn more! What is Infiniband?
- Learn more! Mellanox Cloud and Database Acceleration Solution over SMB 3.0
Ready to Build Your Next SAN?
Be sure to bookmark this page and leverage the step-by-step resources in this article for building "Your Next SAN" with Windows Server 2012 R2, Storage Spaces and SMB 3.0! Along with these resources, be sure to download our free evaluation kit for Windows Server 2012 R2 so that you have the software bits you need to get started in your lab!
As you move forward in your evaluation of storage technologies included with Windows Server 2012 R2, please share your feedback and results in the comments below!