The Role of APM in Continuous Integration and Continuous Release

Today’s software-defined and -driven business requires fast changes in business models—a phenomenon permeating successful businesses. What this means is almost every company is learning how to make small, rapid changes and adjustments within their business, especially within software systems. As a result, IT is feeling immense pressure to evolve, and the net result is a major uptick in private cloud environments (especially Apprenda, Pivotal Cloud Foundry and Red Hat OpenShift) and public clouds (especially Amazon and Microsoft) within our install base of large enterprises. These new platforms enable faster development, testing and releasing of software.

Enterprise customers are getting much smarter by using automation to drive the software life cycle, including continuous integration, and even experimenting with and using continuous release processes. This is no longer a startup scenario; it’s a strategy for our large, enterprise customers. These companies are trying to build agile development and operations teams by changing people, process and technology to enable a DevOps feedback loop:

Wikipedia describes the continuous delivery loop as follows:

Within this loop, application performance management (APM) can be part of build, unit, regression and load tests to drive the automated acceptance testing before new code is pushed to production. In fact, at the recent AppSphere 2015 user conference we had several great talks on this subject.

The first talk was presented by our customer The Container Store‘s August Azzarello, who discussed, “How The Container Store uses AppDynamics in the Development Lifecycle.” The company began with APM in production, in an effort to improve software quality before production. It began the expansion by installing APM within test, integrating it into functional and performance test suites. It also enabled alerting from dev/test so that developers and testers both understood when there were performance deviations and issues before production.

Key features the company leveraged in pre-production included dashboards such as this one:

Comparative analysis views such as this one:

And being able to understand between releases if they were improving or degrading:

The Container Store is using open source tools including Selenium for functional testing and Locust.io for performance testing.

During his discussion, August also explained some of the major benefits the company saw from implementing APM in pre-production. Here are some best practices outlined by August in his discussion:

Monitor everything
Test continuously
Performance test early in development life cycle
Empower development & QA team members

Some benefits The Container Store has seen, according to August, are:

Set performance expectations before production deployments – about 40 percent improvement since it started using AppDynamics in test
Fine-tune alert and business transaction policies prior to production deployments
Identify testing requirements and testing gaps
Decrease performance test result reporting to 20 minutes from five hours

One of my favorite talks (which is why I selected it for the deep dive track at AppSphere) was the talk given by one of our senior sales engineers Steve Sturtevant, who came to AppDynamics from PayPal two years ago. In his talk, titled, “DevOps and Agile: AppDynamics in Continuous Integration Environments,” he explained the integration work he did. He also delved into the questions, Can you scale and auto implement monitoring in dynamic environments? Do you quantify the impact of change? Steve also offered a demo with integration between Puppet, Jenkins, ServiceNow and AppDynamics in his examples, with automation throughout the life cycle.

One of our partners Electric Cloud specializes in continuous delivery and embraces an open ecosystem of technology providers and open source software. It has put together some great resources for people looking to explore the rest of the feedback loop: