I recently had a conversation with some SOX IT auditors and was asked what I meant when I said we were a DevOps shop. I stumbled a little, torn somewhere between an overly simplistic answer and a desire to baptize the unholy. I started by talking about collaboration between the developers and the admins. Mentioning that many of our developers are on call and have sudo access raised some eyebrows. Using Puppet made them pretty happy.

Thinking back on it, what I realize is that for me devops isn’t a strictly defined thing, it’s a mix of operational changes that draw on the best of any tool, without prejudice. We do what works. We do not do what does not work. If we are doing something, and it ceases to work, we stop doing it. I don’t know if that’s devops, but it’s what we do. And it works. So we’re going to keep doing it.

Configuration as code

This is not news, central and key to successful devops is controlling system configuration and change management with code, not with disparate manually managed text files. Here’s the thing, you cannot operate at scale without it. You also cannot change direction, or iterate through rapid change without it. You can rapidly, accurately, and verifiably push changes, stand up environments, or demonstrate a fact to an auditor with just a few keystrokes.

To be sure, it’s a helluva lot easier to start a business on these principles than to migrate after your business is a real thing. However it’s worth the effort.

Sysadmins are our friends, Developers are on call

Most places I’ve worked, neither of those statements were strictly true. At Craftsy, both are. The relationship we foster is one of open collaboration. I’ve never been somewhere where the sysadmins were jerks enough to really justify the animosity they received from dev, and I’ve never met a developer so hopelessly irresponsible to really justify the vitriolic mistrust dealt out by the SA team.

Give your developers sudo and get them in the oncall rotation. Now, they share a burden with your SA group, and a responsibility to the systems their code runs on. Figure out a sensible rotation and escalation, but without that shared responsibility you will always struggle with trust between your dev and your ops.

SaaS, Paas, IaaS, CotS, Cloud, OnPrem, Whatev

The punchline is “all of the above”. Never let yourself get married to a particular way of doing things. We run stuff in the cloud. We run stuff locally. We buy software. We rent software. We rent services. We borrow code. We cloud. And whatever comes up next, we’ll try it one way, whichever way seems to make the most sense. If it works, we’ll keep doing it that way.

I’ve heard some reasonable objections to Cloud/*aaS based on security concerns. Safely extending the trust relationship of your LAN isn’t easy. It isn’t impossible either. Tools like the AWS VPC offering, or SSO/IAM systems offer some very flexible approaches to open your environment and leverage the best tool, regardless of how it’s made available.

Do what works

I’m not sure exactly what the SOX guys thought of my ideas around devops. IT, much like accounting, is an area that traditionally is slow to adopt new approaches. It’s a counterintuitive truth that an industry so deeply engaged with technology is simultaneously so awkwardly regimented in its ways. The standard answer, that change has to come slowly to maintain stability is unsatisfying. Indeed, it belies the fundamental failings of the old methods.

DevOps sets the table for you to achieve stability in any environment precisely by engaging deeply the technology around you.