To be successful in today’s fast-paced business world, IT must develop the ability to consume change at the rate of business while providing stable operations 24×7. Achieving this is more easily said than done, as I discussed in my first blog on why mastering a strong, stable IT operation is critical to becoming agile.
The first step is to stabilize IT operations. That means gaining the ability to quickly enable operational change while mitigating as much risk as possible. To do this effectively requires greater visibility into how the business is consuming IT assets and services. A mature configuration management database (CMDB) is the secret to unearthing this visibility. The CMDB serves as a single source of truth to manage the relationships between IT assets-from data center, network, virtual storage, etc., all the way up through business capabilities.
But like everything in IT, the move to a stable environment involves more than deploying new tools. We had to develop a new culture, improve skills, embrace automation, and create new policies and processes. We had to move from a reactionary mindset to one focused on proactive, non-disruptive service delivery.
Below are seven lessons NetApp IT learned as it transformed its approach and methodology to enable infrastructure operational stability:
- Adopt an integrated IT workflow management platform. NetApp IT had separate workflows for incident management, root cause analysis, change management, infrastructure delivery, etc. Each process varied in maturity and was managed as a silo. To solve this, we adopted a central platform that integrated the separate processes into a single holistic workflow. Productivity improved as the time spent coordinating the hand-off of tasks between silos was eliminated. IT took a huge step forward in delivering IT as a service.
- Evolve the use of the configuration management database (CMDB) to serve as a business relationship management tool. An accurate CMDB is essential to understanding an entire IT environment. Our CMDB has evolved from a simple system for infrastructure asset tracking (physical and virtual) into a tool that provides visibility all the way up to business use. We integrated OnCommand® Insight (OCI) with the CMDB to map virtual storage to its corresponding business services. This approach enables us to analyze consumption and then adjust operations as needed to meet service-level agreements (SLAs). Business relationship managers use the data to help business customers understand how they use IT services in business terms. The CMDB is the single source of truth for managing our IT environment.
- Embrace a rigorous change management process. IT’s challenge is to facilitate change as fast as the business requires, without disruption. To do this we had to focus on three areas: people, process, and technology.
- People. We named a person to be accountable for change management (process, governance, adherence, etc.). Then we identified the right level of people to be part of the standing approval process. This ensures the right people are involved in the process at the right time, all changes requests are centrally managed, and the process remains efficient.
- Process. We developed a process that is supported by our workflow management platform. Our approval cadence is governed by a clear policy that is driven by business needs. By enabling a process that was integrated into our daily IT operations, we were able to move from a 10-day review cycle to a daily review.
- Technology. We used technology and tools to support this streamlined process. The CMDB delivers end-to-end visibility by serving as our single source of truth. OnCommand Insight updates the CMDB with daily discovery information about our virtual storage layer.
With visibility and understanding of our IT environment from top to bottom and availability of the most current information in one place, we can perform risk assessments and system implementations, and make other decisions with minimal risk to daily business operations. More importantly, we significantly increased the rate at which we can consume change while remaining operationally stable. Over time, we proved we could handle more change with minimal impact even as our budget shrank.
- Use monitoring to manage and maintain a proactive operation. We have embraced a combination of proactive monitoring and reactive alerts to help maintain operational stability.
- Proactive: We define and manage thresholds with pre-set operational tolerances, then use a series of dashboards to manage those thresholds at an aggregate level. Our Help Desk/Command Center monitors the thresholds and proactively addresses potential issues before they generate a reactive alert and impact the business.
- Reactive: Because most potential performance issues are addressed through proactive monitoring, we have relatively few reactionary alerts. Admins do not receive informational alerting. This creates a higher level of confidence in alert notifications with the admin team. They now know when they receive an alert that it is both urgent and requires immediate action.
- Automate, automate, automate! In order for us to do more with less we have to rely on automation to remove people from repeatable daily activities or processes. The more we remove manual involvement from repetitive activities, the more our groups can focus on proactive activities that add value. First and foremost, automation delivers predictability by removing the risks of human error. It also offers other advantages: faster time to delivery, process consistency, a common user experience, and smoother hand-offs of tasks between groups. We have found that automation isn’t a one-time deal; we are continually analyzing all our processes for that opportunity.
- Embrace standards over customization. One of NetApp IT’s big mantras is standardization. Not just of physical and virtual assets, but of configurations and processes. Standards are applied to all aspects of how we run IT, including:
- Designing and planned use of physical environments, such as data centers
- Configuring, installing, and deploying virtual/physical assets
- Monitoring criteria of daily operations (in the public and private clouds)
- Administering lifecycle management
- Developing and testing business applications
- Implementing business continuity/disaster recovery plans
Having standards for our entire operations is the foundational cornerstone for operational stability. By adopting standardized practices, we are able to address about 90% of the technology delivery requirements and have provided 99+% stability on the operational side.
- Look beyond infrastructure. As we began our drive toward greater IT stability, we focused on automating our infrastructure elements, including compute, storage, and network assets. Today we are pursuing development of the application layers of IT, including Platform as a Service (PaaS), a push-button service that will encompass the network layer through the application technology layer. Automating the entire stack will enable a faster delivery of business capabilities and add a new level of agility for IT.
Agility Builds on Stability
Operational stability, especially in infrastructure, is a big first step to achieving business agility and delivering IT as a service. In the past year, NetApp IT has achieved a new level of stability by adopting its best practices and maturing the organization. This required moving from a reactive to a proactive mindset and a willingness to look at new ways to leverage technology, standards, and automation to support business needs.
The seven lessons described above helped us gain the ability to stop worrying about our infrastructure and look for new ways to consume change at the rate of business. This trend should continue as we face our future challenges, including adoption of the hybrid cloud and PaaS. It will also help IT face its biggest challenge: recognizing that customer success and business value are achieved when we put our business customers first.
The NetApp-on-NetApp blog series features advice from subject matter experts from NetApp IT who share their real-world experiences using NetApp’s industry-leading storage solutions to support business goals. Want to view learn more about the program? Visit www.NetAppIT.com.