As a service provider to the NetApp enterprise, we continually review and improve the quality of IT services we provide. Using both statistical and anecdotal evidence, we focus on quality, cost, and speed of delivery. As a leader in Business Application Delivery at NetApp, driving innovative improvements that increase our efficiency and effectiveness is a primary concern.
Automating the regression test suite is an opportunity to positively influence quality, time, and cost. Regression testing verifies that software previously developed and tested still performs correctly after being changed and has not introduced new faults. With automation, existing application features and functionality are tested with minimal test team involvement. Scripts can be programmed to radically increase parallelism in test cases and scenarios. This results in reduced test-cycle duration, reduced dependency on human resources, and therefore reduced cost, all without compromising quality.
As an enterprise IT organization, we recognize the depth and breadth of both our responsibility and potential. Our ability to predictably deliver new solutions, features, and functionality into the hands of NetApp leaders, internal users, and external customers and partners is critical to:
- Enable strategic innovation
- Anticipate and adjust as macro business climates change
- Differentiate NetApp in the marketplace and facilitate successful business outcomes
Automating Best Practices
Test automation had been attempted in pockets of IT before with varying degrees of success. A different approach was needed to ensure sustained success on a broad scale. Key areas to be addressed were accountability, standards, and resources.
In launching this new approach, we anointed a small team within Service Quality & Enablement-a move we deemed critical to a successful and lasting effort. Of particular importance was the team’s leader, who would have ownership and accountability. This team’s charter is to monitor, measure, and improve Service Quality. Its members display varied breadth and depth of techno-functional expertise. With test automation recognized as such a high-impact opportunity, building the practice was the team’s first focus.
The development of guidelines and standards was also a key area of focus. In test automation, just as in any development effort, adherence to best practice concepts such as simplicity, portability, and reusability is expected. Making use of the same version control tool used by the broader IT organization, the team put in place a mature source code control strategy. The team summarily raised the bar and expectations around test automation scripting.
Finally, working with NetApp IT’s Strategic Innovation Lab, the team used enterprise PaaS (platform-as-a-service) concepts to create a stable, predictable platform. This platform hosted test-related tools and software in order to enable focus and maturity in test automation practices. Recently, the team completed the migration of the NetApp Test Automation Center (NTAC) from the Innovation Lab to the non-production data center. NTAC is now in use as a fully integrated and key component of the continuous integration/deployment (CI/CD) strategy within NetApp IT.
Executing an end-to-end automation strategy, multiple regression suite automation efforts are now underway for some of NetApp IT’s Tier-1 applications supporting critical business capabilities. We’re learning and adjusting as we go, knowing we’ll continually evolve as we traverse the maturity curve. Well beyond proof of concept at this point, our real-world evidence is extremely positive.
For example, scripts written to perform regression testing of the first module within our customer relationship management system reduced manual human effort by 96% and test execution duration by 77%. Scripting of the second module is nearly complete with similar results expected.
Another highly valuable use case we’ve been able to address is the application sanity test. Typically, after the build and deployment of a non-production application environment, a human from the team (developer, technologist, or tester) was required to validate the integrity of the system. This took resources away from other activities and could potentially delay the use of an environment until someone was available to confirm it. Now, as part of a unified CI/CD strategy, the sanity testing via NTAC happens automatically and immediately. This allows delivery teams to proceed with increased speed and agility.
Using NetApp IT standards, enabling an end-to-end automation strategy and ensuring clarity and accountability have been paramount to our early success. As NetApp IT’s Business Application Delivery organization traverses a very deliberate path toward the hybrid cloud, delivery velocity is increasing and application location is no longer static. These shifts are creating new and increasing challenges with test consistency and conformance. While our immediate focus is to introduce test automation for our suite of business applications, we are already looking ahead at how NetApp’s Data Fabric will help manage and innovate through the challenges.
Going forward, improving the quality, cost, and speed of delivery of IT Services will be enabled by the NetApp Test Automation Center.
The NetApp-on-NetApp blog series features advice from subject matter experts from NetApp IT who share their real-world experiences using NetApp’s industry-leading storage solutions to support business goals. Want to learn more about the program? Visit www.NetAppIT.com.