Originally, this series was just going to be four articles on the DevSec side of DevSecOps. There are many reasons for this, but primarily because that side is cleaner. The other reason is that these topics are beyond the work we were doing at Accelerated Strategies Group.

But we’ve had a number of requests to continue, and I’ve used many tools in the space, so on we go.

Intrusion Detection

Intrusion detection (IDS) started because attackers were getting through firewalls, and, once inside, seemed to have free rein. Intrusion detection wanted to offer another option for detecting and stopping attackers by watching things inside the firewall for indications your organization had been compromised. Relatively quickly, intrusion detection was supplemented by intrusion prevention (IPS). The idea of prevention was to proactively make changes based upon known issues to keep intruders out. Intrusion detection offered information about what paths attackers were exploiting, and intrusion prevention would plug them. Note that IDS and IPS are often further subdivided into host-based (HIDS) or network-based (NIDS) systems. Since their role – detection of intruders – is the same, for this overview we will discuss them together. A serious deep dive into IDS/IPS should look at HIDS and NIDS separately.

In the DevSecOps space, some have folded these into security monitoring, but they are still performing the same basic tasks – just much better. In fact, if you found intrusion detection to be limited and clunky to work with because of rules maintenance and fingerprinting issues, it’s probably time to take another look. Here’s why:

Problems With Traditional Intrusion Detection

The problems with pure rules-based intrusion detection were two-fold.

First, rules were hard and fast.

“Look for X and Y and Z. If all are present, raise an incident.” This tended to result in only two types of rules – ones that were too narrow, and thus brittle, or ones that were too broad and that buried IT/security staff in alerts.

Second, those rules were outdated before they hit the network.

The old adage “You are always fighting the last war,” (meaning you are doing what worked the last time you won) is more true in IT security than almost anywhere else. For a rule or a fingerprint to be assembled, it has to have been seen. That means we were perpetually protecting against previously successful attacks and doing little against emerging threats. Security vendors took a lot of steps before coming to our current solution, which is a great step in the right direction.

Attacks against IT infrastructure share a lot of commonalities. Purely rule-based systems counted on tribal knowledge to code those commonalities into rules. It was clunky and imperfect. Templates helped, by adding common items based upon the type of rule you were creating, but, like rules, they tended to be static and old news.

Enter Machine Learning

Bright people with an eye on the ever-changing environment knew that we needed something that could adapt to multiple attack forms/vectors and draw the type of conclusions that security analysts were tasked with. The ability to reduce the number of alerts that get raised is pretty close to miraculous. Anyone with a web presence sees thousands of security events a day, as attack bots and scanners automate chunks of the attack matrix. Some kind of solution to weed out the false information is nearly mandatory, as no organization has the manpower to monitor at those levels and reasonably follow up on threats.

Machine learning (ML) makes connections between alerts – that might not be a big deal in isolation – and other events that, together, make a case for human intervention. Added to fingerprinting and rules, it is a powerful addition. While a successful attack would require going through rules and making certain you covered the behavior/changes found in the new attack, ML can take that information in and adapt automatically.

This is cranking through volumes of data we couldn’t even imagine a decade ago, and finding anomalous behavior, correlating it and alerting where necessary. How much it reports up the chain is generally configurable, allowing an organization to tighten the flow of alerts that analysts have to deal with.

Go With the Flow

More recently, flow evaluation has become a positive step in the monitoring process. If this connection/user moves from A to B to C, it might be an attack. This allows for another way to rate suspicious activity and raise it to an analyst’s awareness.

An increasing number of vendors are moving the analysis portion of these activities into their hosted environment. This is a “great strength/great weakness” solution – putting the data points gathered from your system into their cloud allows them to correlate events and improve all customers’ responses. The other side of the coin is that you are paying them operational costs each month, and for most vendors, you’re charged based on how much of the service you use. This has budget implications. It also assumes you have connectivity to their service. If you don’t, for any reason, no analysis will be done on collected data. A careful rollout and contingency planning are in order for this scenario to be successful.

Integration with security information and event management (SIEM) systems is ubiquitous. We’ll talk about SIEM next; I just wanted to include here that IDS/IPS vendors are feeding SIEM.

IDS isn’t perfect, and there are a ton of other things out there to help, but it’s a step in the process, one you should consider as you gear up your DevSecOps to professional levels. Some of you are using the tools, but don’t have them integrated into your DevOps systems. This is a nudge to do so. If resolution of an attack automatically creates tickets for Dev to make the code (or ops to make the config) more secure, it absolutely is a plus to the organization responding proactively.

Keep kicking it. These are just tools, and they’re useless without you. Figure out what your organization needs, and update those needs regularly as the environment (both attackers and architecture) continually changes around you.