This week I am attending the FAST ’15 Usenix conference in Santa Clara. One of the sessions I sat in on yesterday was a discussion of Software-defined Storage (SDS), and how it compares and contrasts with Software-defined Networking (SDN). This got me to thinking about the future of SDS, and whether or not it will deliver the promise of true enterprise-wide storage management.
Fundamentally, SDN and SDS use the same framework. A software “control plane” is created to manage a group of similar devices residing on a “data plane”. As you’d expect, the control plane is responsible for sending control commands to the devices, which comprise a data plane that moves data between points across the server/networking/storage infrastructure. Ideally, devices on the data plane should be fast and dumb, since all the intelligence has been moved upstream to the control plane. Any attempt to make independent decisions at the device layer would just interfere with decisions made at the control layer. Removing intelligence from the devices allows the control plane to act as the Uber authority that can better manage data across all devices, from its higher-level perspective.
The concept of the control plane/data plane works well for routing ethernet packets, which carry a universal format and a header with known attributes i.e. source IP, source port, destination IP, destination port, and protocol. Given this information, the SDN controller can easily create master routing tables and modify them on-the-fly as network bottlenecks come and go, and as network ports are added or removed. No vendor cooperation is needed, and no new networking standards need to be developed in order for SDN to work.
Carrying SDN into SDS
SDS, however, is challenged when it comes to this model, primarily because there is very little standardization between storage vendors. Case in point is EMC, one of the main proponents of SDS. The folks at EMC have been pushing hard for SDS, and publishing diagrams such as the one below that shows an idealistic view of the SDS model:
According to the EMC pitch, the ViPR SDS model was designed with an any-to-any model in mind. Unfortunately, reality does not align with this model. According to storage industry contributor Dave Raffo:
“One of the selling points of ViPR that EMC has pushed since launch was that it would work with third-party and commodity hardware…While ViPR has some level of support for NetApp, Hitachi Data System and IBM XIV arrays, it remains slanted far towards EMC platforms. Of course, that is no surprise. EMC didn’t develop ViPR to move customers to other vendors’ arrays.”
Therein lies the problem. What EMC (and others pushing SDS) have come to realize is that there is no magic method in harmoniously controlling heterogeneous storage devices. Each storage array has its own storage OS, its own API and its own a set of capabilities. In order to accomplish what EMC envisions, every storage array in its SDS network would need to be detuned down to the lowest common denominator, with all of the storage management features built into the software control layer. This seems unrealistic to me, as it would come at incredible expense, be subject to the whims of storage array software upgrades, and customers would balk at having their expensive storage arrays reduced to the intelligence level of JBOD (Just a Bunch of Disks) racks.
Any Other Options?
Speaking of JBOD – what about JBOD? Why not roll those expensive storage arrays out to the shipping dock, and replace them with racks and rack of commodity storage devices, dumb and fast, just like SDS wants? Well, because no software SDS vendor appears to be remotely close to providing the built-in features that enterprise storage admins have come to expect – things like self-healing, synchronous replication, cache tiering, nondisruptive upgrades, and performance monitoring, to name a few. Again, this would come at tremendous development costs with no guarantee that customers would adopt. There are simply no products on the horizon that seem to be able to deliver this capability.
Open source you say? Yes, of course, there is OpenStack, and fine work being done in developing Cinder (block storage) and Swift (object storage) standards, but those rudimentary interfaces are a far cry from a sophisticated unified command set for all storage arrays. Think SCSI on steroids – that is what would be necessary for true (heterogeneous) SDS.
What Are We Left With?
The good news is that alternatives to SDS aren’t so bad, and have been around for a long time. Established storage vendors have done a pretty good job developing SRM (storage resource management) software. Some attempt to jump on the bandwagon by calling this SDS, but don’t be fooled, homogenous SRM is what they are best at, since they know their own platform architectures better than anyone else. I’m biased towards my company’s SRM products: SANtricity Storage Manager and NetApp OnCommand – but I will admit the others are pretty good at managing their own storage.
My advice – if anyone claims to offer SDS, take it with a grain of salt, and ask them if they offer the exact same capabilities on OPS (other people’s storage) as they do on their own.