IBM Open Sources SysFlow Monitoring Platform

IBM has announced that its SysFlow monitoring platform is now available as an open source project.

Fred Araujo, a research scientist in the Cognitive Cybersecurity Intelligence Group at IBM Research, said IBM developed lightweight SysFlow agent software and monitoring tools as a way to provide more context around the telemetry data being collected while simultaneously reducing the amount of data that needs to be stored.

SysFlow encodes a representation of system activities into a compact format that records how applications interact with their environment, Araujo said, noting that level of context provides deeper visibility in everything from container workloads to cybersecurity forensics. However, unlike existing monitoring platforms, SysFlow doesn’t require IT organizations to collect a massive amount of data to achieve that goal—it is intended to provide for a superset of the NetFlow framework used to analyze network traffic patterns to capture system events, he said.

Araujo noted IBM doesn’t envision SysFlow eliminating the need for legacy log analytics platforms, as they provide a way to analyze log data. However, SysFlow does enable IT organizations to apply analytics via a graph-like visualization to surface patterns that goes beyond a comparative simple rules-based approach, said Araujo. For example, SysFlow’s approach will make it easier to uncover the relationship between various events that make up a cybersecurity attack and subsequently to identify what countermeasures to employ to create the appropriate kill chain response. It also should substantially reduce the amount of fatigue cybersecurity teams experience from chasing down false-positive alerts, he said.

SysFlow is designed from the ground up to integrate with both open source frameworks such as Apache Spark and commercial analytics platforms via an open serialization format and associated libraries. IT organizations also can leverage a set of reusable components and APIs to make it easier to deploy telemetry probes. IT organizations also can take advantage of an extensible policy engine that can ingest customizable security policies described in a declarative input language, which then can be checked against records captured by SysFlow.

Araujo said IBM developed SysFlow to address the challenges associated with managing application workloads in public clouds, but the framework can be applied to any on-premises IT environment to create a single pane of glass through which multiple platforms can be monitored.

Regardless of the platform on which a workload is deployed, Araujo said as more responsibility for managing and securing applications shifts left toward developers in the age of DevOps, those teams need access to more sophisticated tools that are readily available and simple to deploy. There are several open source initiatives underway to provide that visibility, and one day many of them might coalesce into a single initiative.

In the meantime, as the dependencies between applications and IT infrastructure continue to increase, it’s clear IT monitoring as a core part of any set of best DevOps practices is now an absolute requirement to achieve and maintain observability.

Mike Vizard

Mike Vizard

Mike Vizard is a seasoned IT journalist with over 25 years of experience. He also contributed to IT Business Edge, Channel Insider, Baseline and a variety of other IT titles. Previously, Vizard was the editorial director for Ziff-Davis Enterprise as well as Editor-in-Chief for CRN and InfoWorld.

Recent Posts

5 Reasons Why Your App Needs an Error Monitoring System

All software applications have errors. Bugs are simply part of software development. That’s why engineering organizations need to know exactly…

3 hours ago

Pega Low-Code Platform on Display at PegaWorld iNspire

Low code enables developers to create applications using visual tools and models. Developers looking for resources to improve their productivity…

2 days ago

Report: Debugging Efforts Cost Companies $61B Annually

Undo, a provider of a software failure replay platform, this week published a report in collaboration with a Cambridge Judge…

3 days ago

Report Finds Most Cloud Security Issues Left Unaddressed

Accurics, a provider of a platform for assessing the security of cloud computing environments, has published a report that finds…

3 days ago

Survey Sees Increased Agile Adoption During COVID-19

A report published by, the parent company of CollabNet VersionOne, finds 43% of organizations have increased their reliance on…

3 days ago

Cisco Acquires ThousandEyes to Advance Network Visibility

Cisco Systems this week announced its intent to acquire ThousandEyes, provider of an internet monitoring platform that Cisco plans to…

3 days ago