Google and Netflix today jointly announced an open-source automated canary analysis service, dubbed Kayenta, that applies statistical analytics to optimize DevOps processes.
Andrew Phillips, a product manager for Google Cloud Platform (GCP), said goal is to make it easier to consume advanced analytics by integrating Kayenta with the open source Spinnaker continuous delivery pipeline platform developed by Google. However, because Kayenta is based on software developed by Netflix, an IT organization can feed data from any cloud or on-premises IT environment into the canary analysis service, said Phillips. Kayenta also can ingest data from a broad range of monitong tools, including Stackdriver, Prometheus and Datadog.
Phillips said the goal is to continually iterate on Kayenta to make it simple enough for almost any organization to take advantage of canary analysis, which enables DevOps teams to track metrics such as memory usage of a critical component. Kayenta fetches user-configured metrics from their sources, runs statistical tests and provides an aggregate score for the canary. Kayenta can automatically promote or fail “the canary” or trigger a request for human approval based on those test results.
Specifically, Kayenta creates a pair of control/experiment time series datasets and calls a canary judge, which performs statistical tests, evaluating each metric individually and returns an aggregate score from 0 to 100 using preconfigured metric weights. Depending on the configuration, the score is then classified as “success,” “marginal” or “failure.” Success promotes the canary and continues the deployment, a marginal score triggers a human approval path, and a failure triggers a roll back.
The basic canary concept involves routing a small subset of production traffic through a deployment that reflects the changes (the canary) and a newly deployed instance that has the same code and configuration as production environment, which serves as the baseline for testing. The analytics engine then compares the metrics between the canary and the baseline to identify any significant impacts the changes might have on the production environment. Kayenta in effect provides a mechanism for modeling risks based on proposed changes to the production environment, said Phillips.
Google and Netflix expect third-party vendors will soon be incorporating Kayenta into a variety of DevOps tools and cloud platforms. In the meantime. Google views Kayenta as one more way of enticing DevOps teams to take advantage of Spinnaker to manage their DevOps pipelines in the cloud.
Historically, setting up a canary analysis system has been a complex endeavor requiring significant engineering skills. Google is moving to democratize canary analysis by making it available as a service that can be invoked on demand, said Phillips.
Phillips said Kayenta provides access to data that can be used to drive a wide range of predictive analytics applications that can be used to automate DevOps processes. It will be a while before Kayenta data is used to drive artificial intelligence (AI) models to automate those DevOps processes.
Given the rate at which applications are being added and updated in production environment relying on manual testing is quickly becoming unfeasible. The days when DevOps teams needed to manually create their own statistical analytics models are clearly coming to a merciful end.