During your teenage years, you may have figured out a way to get a ‘yes’ from your parents by playing off of the responsibility and knowledge gap between them…walking up to your mom and saying something like, “Hey Mom, Dad said he would drive me, can I go out tonight?” and then proceed to your dad and say something like, “Hey Dad, Mom said I can go out tonight and you need to drive me.” There is nothing cleverer than a determined teenager wanting to get his or her way. Such is the case of targeted attackers and the way they play one IT department’s responsibilities off another. This is the grey area where the organizational chart and separation of duty can introduce weaknesses exploited by the more advanced threats we face today.
You don’t have to look far into your organization to see this at work. The security team does its thing, networking engineers do their thing, and IT operations teams in general all have administrative realms that define functional boundaries as to what they can see and do. To borrow from Colonel John Boyd’s OODA loop, each team has its observation and orientation (OO) process for gaining operational visibility and making decisions, as well as specific decisions and actions (DA) they can execute within their administrative realm or set of capabilities to bring change to their organization. The weakness I’m highlighting in this post is when the decision and action appropriate for a set of observations, and the observations themselves, exist in two separate departments and can be exploited by the attacker.
To exemplify the problem, let’s begin with the question: Is a distributed denial-of-service (DDoS) attack a security or a network engineering issue? The problem I see most often is that one department will have a precise view into network telemetry for early threat detection, but will not yet have the ability to mitigate or remediate the threat.
Or how about the situation where credentials for one of your executives are harvested, and now the attacker need not trigger any security events at all in order to carry out an attack. In this situation, the attacker merely needs to log in and plan the next stage of the attack, roaming freely and without suspicion across your network because they are only generating networking traffic. The network team does not act because everything is still available, and the security team does not act because there is nothing being denied in access logs, no firewall violations, and nothing triggered on the IDS/IPS.
Let me be clear in that I am not saying that everyone has to become an expert in everything; specialization is healthy, but only when it does not compromise communication! In the example of Mom and Dad being played by the teenager, if Mom and Dad had a better communication protocol, the teenager’s tactics would be thwarted. If you look at some of your operational procedures, and start to think like an attacker (or teenager again), I’m sure you will find at least half a dozen communication gaps that a clever attacker could exploit.
This organizational dysfunction is going to require change that may appear to be radical because the systems that support it are so ingrained. Big analyst firms have defined their technology categories as law and they police them as if you were breaking the law if you dare to create something different – something more cross functional. Some best practices out there are, in my opinion, worst practices because they are so focused on the operational effectiveness of the department, and they leave out other departments or the active threat entirely. Lastly, even some prescriptive compliance requirements force organizations into strict separation of duties, but when implemented, create a broken OODA loop where the ‘OO’ does not connect with the ‘DA.’
There is an opportunity for DevOps to completely recalibrate how we deliver a cross functional operational unit which is why I took the time to write this article for Devops.com. Through the lens of Boyd’s OODA loop, you can already see the security ramifications because the tempo of DevOps allows the defense to loop at a faster rate than the adversary – the dominant strategy of the OODA Loop.
I hope this post is a call to action for all you big thinkers. Get creative, get innovative, and do not be afraid to erase some lines and boundaries within your IT organization if it fosters a higher level of communication amongst your teams. Think of your organization like a great band playing live music. All of the performers play different instruments, yet they are listening to one another just as hard if not harder than they are producing sound. They improvise and adapt to one another; monitoring systems are put in place so they can have just enough of each sound in their own ear. This is what effective, cross-functional teams look like. I’m just going to say it: DevOps is Rock and Roll!
Oh, and hey, follow me on twitter: @tkeanini
