Testing Emerges as DevOps Hurdle in Mainframe Shops

A survey of 400 senior IT leaders responsible for application development using mainframes finds 90% believe automating more test cases could be the most important factor in their success, as the pressure to accelerate innovation increases. However, the same survey finds only 7% of organizations automate the execution of test cases involving mainframe code today.

The survey, which was conducted by Vanson Bourne on behalf of Compuware, also finds the time required to conduct thorough testing is the biggest barrier to integrating the mainframe into agile and DevOps processes for just over the half the survey respondents (53%), followed by a lack of skills (39%) and lack of understanding how code changes will impact distributed applications (34%).

A total of 82% said that unless they can automate more test cases, they will not be able to meet the need for speed the business now requires. That may account, in part, for why 56% of respondents said they plan to start automating the execution of test cases on mainframe code within the coming year.

The biggest benefits expected to be derived from those investments include greater efficiency within the IT team (59%), improved consistency of testing (53% ), reduced chance of human error (47%), reduced pressure on IT teams (47%), accelerated innovation cycles (46%) and less chance of bugs making it into production (40%).

Primary areas of focus for automated testing are ranked as functional testing (63%), followed by integration testing (53%) and unit testing (37%).

The survey also finds development teams are spending, on average, more than half their time (51%) on testing during the release of a new mainframe application, feature or function. That compares to 39% of their time testing applications, features or functions on applications that don’t run on mainframes.

A full 92% said mainframe teams are spending more time testing code than they needed to in the past because of the growing complexity of cross-platform application environments. Issues organizations are contending with are “Manual and time-consuming test processes” (44%), “Mainframe testing processes require specialists that are not always readily available” (40%), “Testing processes are costly” (39%) and “It’s hard to visualize and understand dependencies” (35%).

Not surprisingly, 77% of respondents find it increasingly difficult to simultaneously increase quality, velocity and efficiency to meet business innovation goals when developing and testing mainframe application code. Top concerns when it comes to mainframe development are introduction of security flaws (50%), impact on customer experience (38%) and disruption to operations (36%).

A full 85% said it is becoming more difficult to deliver innovative and faster applications without compromising quality or increasing the risk of bugs finding their way into production.

Steen Brahe, product manager at Compuware, said organizations that have mainframes are usually under pressure to innovate faster now because competition from startups relying on DevOps processes and modern platforms has increased sharply. Rather than incurring the cost of replacing mainframe applications on which the business currently depends, organizations that have mainframes are looking to extend DevOps processes to integrate emerging and legacy applications.

It’s not clear to what degree those organizations will succeed. However, the one thing that is clear is organizations with mainframes that fail to embrace best DevOps processes are already most certainly doomed.

— Mike Vizard