At the recent DevOps Enterprise Summit, we had a chance to sit down with Justin Arbuckle and discuss his concept of “compliance as code” and why it’s so important that security and compliance tests be built into DevOps practices at large enterprises. Recently, Arbuckle was chief architect at GE Capital where he introduced and sponsored DevOps at that organization – so he certainly knows about what it takes to “do DevOps” in large, highly regulated environments.
Currently, Arbuckle is the vice president of EMEA at CHEF and also chief enterprise architect there. If you watch his talk from the DevOps Enterprise Summit, Hunting the DevOps Whale, you’ll see that he is passionate about the transformative potential of DevOps in large enterprises.
At CHEF, Arbuckle is in helping large enterprises understand how to “de-risk” their migration to cloud with automation. We caught up with him at the DevOps Enterprise Summit to ask why that is so important. Here’s what he had to say.
DevOps.com: When it comes to DevOps and regulatory and security concerns, many people say, “Well, just automate those tests.” What does that mean, to you? How do you “automate in” security and compliance?
Arbuckle: I think, in terms of the specific issue of regulatory compliance, a lot of what we think of as the being compliant process today is a complete myth. There is just so much detail that needs to be checked that the organization is always in a position where they have to trade off against, “It’s good enough, we’re ready to go” versus, “We’re not going to go anywhere until we’ve literally crossed every T and dotted every I.”
The reality is that a lot of the regulations, Dodd-Frank is a good example, are getting to a level of detail where they’re beginning to talk about process. They’re beginning to talk about how you design product. They’re getting very deep indeed.
So the organization that has to govern that, can’t do it based on just this idea of oversight at particular points in the life cycle. Because if you’re going to do it properly you literally slow your velocity to zero, and so you’re making a trade-off between compliance and competitiveness.
The way to break that cycle is to, and this is a big paradigm shift for regulators, a big paradigm shift for compliance people inside organizations, you have to start being able to express those compliance requirements in a form that’s codable. Using something like Chef for instance what we’re able to do is we’re able to specify a particular policy in a way that is automatable. In a way that we know is always going to be implemented every single time.
DevOps.com: Would that include input from security people, compliance teams, design teams, system architects?
Arbuckle: Absolutely. Honestly, in my experience I think the number of organizations that can count fully detailed, fully implementable — and that’s the key word — security policy by their infrastructure people is countable on one hand.
Generally speaking, there are many obvious guidelines that are standards-based. But as the security organization tries to keep up with security threats, they’re obviously learning as they go. In this way policy always lags. And the only way for the organization to catch it is through this process of documentation, policy, and checks. And through it all they know that the standard is nonsense because it’s out of date by definition. So they have to create a point-in-time review. And these point in time reviews bring velocity to a halt.
DevOps.com: Yes. Especially when something is wrong. They hit a red button and everything stops coming off the assembly line.
Arbuckle: That’s right. The approach that we have to get to is the idea of policy as code. But you are not going to be able to, on the first go, be able to codify everything. Very often with automation, based on my experience, is that the difficulty at large enterprises has actually less to do with the tools and more to do with knowing what you want to automate. Knowing what you actually want to put in infrastructure as code.
It’s the same thing with security teams. Ask them to go through the most important things that need to be tested in automation, and getting them to go through that process. It’s often difficult because they are not development people typically. They’re not infrastructure people, And they’re not operations people.
They’ve kind of created their own “securocracy” if you like.
What security teams have to do is understand their work is subject to source control as well. Their work is subject to variation and evolution over time and we should write tests to ensure that the following policies are being applied consistently.
DevOps: And I assume these tests are designed over time, as one goes along? For example, if there’s a new malware or regulatory beast that you haven’t seen before, you can design a test and then implement that as part of the life cycle?
Arbuckle: That’s exactly right. And the great thing about tests is they are additive. You’ve got a test that tests for the range of ports that are open on the operating system and then you add another test, maybe testing for the types of applications that are installed on the box. They are two separate tests and they are additive. So as you continue, you build up your library of tests.
And for most organizations going through the adoption of test driven development, as an example, they have to do it that way. A large application that is beginning to develop in an agile way is going to have a lot of tests. If you take a typical large core banking application, for instance, it will probably have something in the region of about 5,000 use cases. That’s thousands and thousands of tests. Let’s just agree that in no short period of time are you going to write those tests.
DevOps.com: That’s an enormous catalog of tests. How do you keep them up to date and fresh? I imagine as security changes, regulatory compliance changes, you may have to tweak these tests as you go along. How do enterprises keep up?
Arbuckle: Like any requirement you have stakeholders that are responsible for those requirements. In just the same way, if you had for instance a mobile application and you have a product owner for that mobile application and that product owner is prioritizing feature sets you should also have a product owner for compliance.
And the product owner should be detailing what the set of compliance requirements are for the product.
Essentially compliance is embodied in the tests that are run against the appropriate applications. And so to manage automation and compliance as code, you need to have a product owner for compliance. And compliance as code, or automation, is the way it has to be in order for compliance to actually be true and consistent, as opposed to a myth.
