Log analytics is all the rage these days among IT and security professionals trying to discover the root cause of any performance issue or security breach. But there’s little consensus about how best to go about analyzing logs. Many IT organizations have made major investments in commercial platforms such as Splunk, while others prefer to rely on the less expensive, open-source tools Elasticsearch, Logstash and Kibana, collectively known as the ELK stack.
As a provider of a cloud service based on the ELK stack, Logz.io has been providing organizations that prefer to rely on open-source software an alternative to deploying and maintaining their own ELK stack. Now Logz.io is extending that service to include Logz.io Application Insights, which employs machine learning algorithms to create a model of normal IT operations that can be used to more easily identify anomalies, and Logz.io Data Optimizer, which helps identify which logs are worth keeping long-term.
Fresh off raising an additional $23 million in capital, Logz.io CEO Tomer Levy says his company is building up a pool of machine data to drive advanced analytics fueled by machine learning algorithms. Those insights then can be shared proactively with other Logz.io customers to head off production issues before they even occur. At present, Logz.io has more than 400 customers.
Levy says the Logz.io Data Optimizer should enable IT organizations to substantially reduce their costs. Most providers of log management tools charge based on the amount of data collected. By helping organizations identify extraneous log data, the cost of implementing analytics of machine data drops considerably, Levy says.
Logz.io Application Insights, meanwhile, provides a simple way to identify the last changes made to the environment that might have created a performance issue or introduced a security vulnerability.
Levy says Logz.io, via application programming interfaces (APIs), can be integrated with multiple continuous integration/continuous development (CI/CD) platforms, including Jenkins. In fact, log analytics is now a fundamental element of any mature set of DevOps processes, he notes.
Platforms for analyzing machine data is one of the fastest growing sectors of enterprise IT. New research from Zion Market Research forecasts that the IT operations analytics market is expected to grow on a 23.7 percent compound annual basis to reach $9.2 billion by 2020.
Most DevOps issues only take a few minutes to resolve. But identifying the root cause of an issue can take weeks, sometimes even months, to identify. In more advanced IT organizations, centers of excellence around log analytics are now being created to identify not only potential IT and security problems, but also issues impacting, for example, the amount of revenue being generated via a specific application.
It still may be a while before log analytics become pervasive across the enterprise. But every event in the IT environment generates a log, so the real challenge and opportunity is correlating all those log events into something that manifests itself as a truly actionable intelligence.
— Mike Vizard
