DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • DevOps Chats
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Communities
    • AWS Community Hub
    • CloudBees
    • IT as Code
    • Rocket on DevOps.com
    • Traceable on DevOps.com
    • Quali on DevOps.com
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Digital Anarchist
  • Media Kit
  • About
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DevSecOps
  • Leadership Suite
  • Practices
  • ROELBOB
  • Low-Code/No-Code
  • IT as Code
  • More Topics
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps

Home » Editorial Calendar » xMatters Sees Value in Continuous Testing

xMatters Sees Value in Continuous Testing

By: David Geer on August 12, 2016 Leave a Comment

xMatters builds an intelligent communications platform that alerts and connects company stakeholders for IT incident response and in manufacturing line workflow support and DevOps systems scenarios. Customers who count on xMatters also count on the effectiveness of the vendor’s continuous testing mechanisms: A failure of continuous testing early on at xMatters could lead to a magnified impact on the customers’ crises response, resulting in costly repercussions.

Recent Posts By David Geer
  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 2
  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 1
  • Sounding the Death Knell for Agile: Not so Fast!
More from David Geer
Related Posts
  • xMatters Sees Value in Continuous Testing
  • DevOps: Shift Left to Reduce Failure
  • xMatters Introduces Advances in Analytics to Speed Efficiency of Incident Response and Resolution
    Related Categories
  • Continuous Testing
  • Editorial Calendar
    Related Topics
  • continuous delivery
  • continuous testing
  • intelligent communications platform
  • regression testing
  • testing automation
  • testing environment
  • xmatters
Show more
Show less

XMatters uses continuous testing to promote continuous delivery of its intelligent communications platform, delivering changes to a limited number of features in an accelerated fashion. “This reduces our risks of failures and makes it easier to resolve defects,” said Deepa Guna, QA Architect at xMatters.

DevOps/Cloud-Native Live! Boston

For xMatters, the proper use and placement of continuous testing in the DevOps pipeline is critical to ensure the intelligent communications solution does not introduce errors during customer crises.

xMatters Software, Languages and Tools

The xMatters platform consists of a service stack architecture with front-end technologies accessible via the web and back-end technologies that support it. According to Guna, the platform takes advantage of Java, RESTful JSON, Node.js and Postgres, which run on data centers that are geographically dispersed in North America, Europe and the Asia Pacific regions.

“Our data centers are built on infrastructure as a service, which includes technologies such as OpenStack, Puppet, Fabric and Consul,” Guna said. XMatters also has a mobile application for iOS and Android platforms as part of its communication service.

xMatters Testing Approaches, Before and After DevOps

The company’s testing before the advent of the continuous delivery pipeline included manual test execution of regression tests in the midst of a slow process of releasing one large set of features per month and deploying customer-reported defects, Guna said.

Today, xMatters uses automated and manual regression tests and unit test code coverage monitoring, releasing small sets of features and resolved defects weekly while enabling customers to provide feedback early in the feature development process.

XMatters uses several tests during the development, test and release phases of the DevOps pipeline and when delivering the platform to the customer. The development phase includes the following testing tools and their associated types of testing:

  • SonarQube analyzes the quality of the code.
  • Jacoco measures the code coverage of the developer unit/integrations tests.
  • Git + Jenkins run the unit and integration tests before xMatters commits the changes.

The testing phase, Guna said, runs tests as follows:

  • Jenkins deploys artifacts in test environments and runs automated tests.
  • An automation framework that includes Jmeter, Gradle, Java, TestNG and Selenium runs its tests.

The release phase tests software after this manner:

  • Jenkins deploys artifacts in test environments and runs automated tests.
  • An automation framework that includes Jmeter, Gradle, Java, TestNG and Selenium runs its tests.
  • Splunk and Takipi do exception monitoring.

When xMatters delivers the platform to its customers, Guna said, it performs the following tests:

  • Runscope monitors the health of the system to ensure critical services are running.
  • Pingdom monitors the status of the customer websites.

Drilling Down into Release Testing

“During the release testing phase, we put the application through 1,000 fully automated and manually initiated tests that we execute with every release cycle. We also have tools that are monitoring for exception during the release testing,” Guna said. Passing this phase successfully, the new software platform release is ready to be pushed into the production environment.

XMatters uses the continuous delivery pipeline to deploy the new platform release first to the customer staging environment and next to its production environment, Guna said. “All production environments have ongoing smoke tests that we execute every five, 10 or 15 minutes, based on customer tier, to monitor the health of the services.”

Results Metrics

xMatters’ method of continuous testing seems to be working for the vendor. “We have reduced the number of major incidents in production due to application defects by 40 percent this year compared to last year,” Guna noted.

Filed Under: Continuous Testing, Editorial Calendar Tagged With: continuous delivery, continuous testing, intelligent communications platform, regression testing, testing automation, testing environment, xmatters

Sponsored Content
Featured eBook
DevOps: Mastering the Human Element

DevOps: Mastering the Human Element

While building constructive culture, engaging workers individually and helping staff avoid burnout have always been organizationally demanding, they are intensified by the continuous, always-on notion of DevOps.  When we think of work burnout, we often think of grueling workloads and deadline pressures. But it also has to do with mismatched ... Read More
« Test-First Development: Processes and Tools for Success
8 Reasons to Go ‘All In’ and Attend Jenkins World »

TechStrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Accelerating Continuous Security With Value Stream Management
Monday, May 23, 2022 - 11:00 am EDT
The Complete Guide to Open Source Licenses 2022
Monday, May 23, 2022 - 3:00 pm EDT
Building a Successful Open Source Program Office
Tuesday, May 24, 2022 - 11:00 am EDT

Latest from DevOps.com

DevSecOps Deluge: Choosing the Right Tools
May 20, 2022 | Gary Robinson
Managing Hardcoded Secrets to Shrink Your Attack Surface 
May 20, 2022 | John Morton
DevOps Institute Releases Upskilling IT 2022 Report 
May 18, 2022 | Natan Solomon
Creating Automated GitHub Bots in Go
May 18, 2022 | Sebastian Spaink
Is Your Future in SaaS? Yes, Except …
May 18, 2022 | Don Macvittie

Get The Top Stories of the Week

  • View DevOps.com Privacy Policy
  • This field is for validation purposes and should be left unchanged.

Download Free eBook

Hybrid Cloud Security 101
New call-to-action

Most Read on DevOps.com

Why Over-Permissive CI/CD Pipelines are an Unnecessary Evil
May 16, 2022 | Vladi Sandler
Apple Allows 50% Fee Rise | @ElonMusk Fans: 70% Fake | Micro...
May 17, 2022 | Richi Jennings
DevOps Institute Releases Upskilling IT 2022 Report 
May 18, 2022 | Natan Solomon
Making DevOps Smoother
May 17, 2022 | Gaurav Belani
Creating Automated GitHub Bots in Go
May 18, 2022 | Sebastian Spaink

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2022 ·Techstrong Group, Inc.All rights reserved.