One of the biggest trade-offs in any data center is power and capacity, the two biggest expenses of any data center. The golden rule is that these two costs increase together-the more racks of hardware you have, the more power you need to run it. This means when you need more capacity, you need more power, which could result in a cooling issue. If you have enough cooling and power, you could run out of rack capacity.

 

NetApp IT was able to address the power and cooling costs in a multitude of ways. We started by making changes to the facility itself. We installed non-traditional raised floors. We introduced overhead cooling, economization, and cold aisle containment over six years ago. These changes have helped control our power and cooling costs.

 

Changing Relationship between Power and Capacity

A NetApp IT data center operation analysis compiled over the past decade shows that the relationship between power and capacity is evolving due to other factors as well. We are seeing that while our compute and storage capabilities are increasing, our power costs have been actually dropping. This shift is due to several reasons: the availability of the cloud, smaller form factors offering more horsepower, and virtualization, among others.

 

The chart illustrates this point. Our power requirements peaked in mid-2011 when we opened a new NetApp production data center, the Hillsboro Data Center (HDC). As we moved operations into HDC and closed other data centers, power consumption dropped while storage and compute increased significantly. Since then we’ve seen this trend continuing.

 

10 Year Data Center Capacity Trends.jpg

 

The following factors are contributing to this change:

  • In the past, each app had its set of hardware and its own power supply, which translated to thousands of servers, an expensive model to maintain. Because of virtualization, the same applications can be hosted on 10 to 20 physical machines in a few racks using around 20 kilowatts (kW). NetApp IT’s compute is 75% virtualized now.
  • All Flash FAS adoption. Our solid-state disks (SSD) take very little power (1-2kW as compared to 5-6kW for traditional disks per rack); our Flash hardware even less. As result, full storage racks aren’t even close to reaching their power limits. Our decision to use Flash for all non-archival storage going forward means even lower power consumption.
  • High-density storage rack design. HDC has high-density, taller-than-normal racks–52U as opposed to traditional 42U or 47U racks with more power (10kW/rack). Hardware that used to take four racks now takes half of a rack, thanks to higher density disks and higher IO capability clusters/filers. This unique design has reduced the number of infrastructure/connection points, shrinking the data center footprint and enabling a build-as-you-grow approach.
  • FlexPod® We have eight FlexPod systems hosting hundreds of applications in a rack connected to the Cisco fabric for networking and compute via the Cisco Unified Computing System™ (UCS). The applications are hosted on thousands of machines, but thanks to virtualization and cloud, that doesn’t mean thousands of physical servers. Most of the applications are hosted on virtual machines. These footprints will continue to shrink as compute core processor power increases, hardware size shrinks, and power consumption requirements fall due to technology advancements.
  • Smart power design. The Starline busway system supports ‘anywhere’ power and connector types, and with our smart layout we can utilize power borrowing that enables us to share power across multiple racks. We can draw power from a parallel busway if a rack needs more than 9kW. We have effectively removed power as a consideration in our hardware installations, especially for compute.

Our analysis shows that the relationship between storage/compute capacity to deliver applications and power will continue, even as we begin to take advantage of the hybrid cloud. Instead of building arrays to meet peak workloads–which translates to idle capacity–we will be able to take advantage of the cloud’s elasticity. This, in turn, will reduce operational, licensing, and management costs in the data center and we’ve seen a reduction in capex spend as well.

 

Adopting a hardware lifecycle management strategy is a key factor in reducing power consumption and improve capacity management. In our HDC migration, we were able to decommission 96 of 163 systems and 40 filers (of 2 PB of storage); more than 1,000 servers were either migrated or decommissioned. The configuration management database (CMDB), NetApp IT’s single source of truth for everything in IT operations, also plays a major role in helping us track, manage, and analyze power and capacity over time.

 

Every data center environment is different. Each company faces its own challenges in controlling its power consumption costs while maximizing its storage and compute. However, as we have seen, adopting a hardware lifecycle management strategy and leveraging innovations in technology and power design can make a significant difference in your data center costs.

 

The NetApp-on-NetApp blog series features advice from subject matter experts from NetApp IT who share their real-world experiences using NetApp’s industry-leading storage solutions to support business goals. Want to view learn more about the program? Visit www.NetAppIT.com.

mm

Randy Egger

As a data center lead, Randy is responsible for NetApp's IT global data center space and power capacity planning. He has local accountability and responsibility for the operations of the NetApp Hillsboro Data Center (HDC) and was heavily involved in its planning, development, and construction. Randy has more than 23 years of experience in high tech, with a focus on data center implementations and projects.