“I, For One, Welcome Our New Robot Overlords,” is the title of Mykel Alvis’ (@mykelalvis) session at the 2016 All Day DevOps Conference. He wasn’t trying to curry favor with the new robot rulers, a lá Kent Brockman, but, instead, was evangelizing on the importance of precision in DevOps.
Mykel is the DevOps Coach at Cotiviti Labs, the development arm of Cotiviti. Every day it handles healthcare data, which as you know happens to be some of the most regulated and protected data around. As Mykel said, “We don’t do cat pictures on the internet.” A mistake, breach, security flaw, etc., could mean the exposure of privacy and financial data for millions of people. But, it is a large organization, with exposure that needs to be managed. It has 200-plus repositories, three to four deployments per day, 15 developers and 0 operators (sort of). Without the right discipline and rigor, costly mistakes could happen.
Hence, Mykel’s need for the robot overlords. As he noted during the conference session, robots have, “Lower utility, but greater predictability.” They do exactly what they are told, and they do it the same way, every time. By contrast, humans are superior to responding to unpredictable events. Mykel’s goal is to leverage the strengths of each. Cotiviti Labs has systems, policies, procedures and a culture in place that supports the automation of tasks as much as possible to ensure mistakes are kept to a minimum and the humans can focus on the unpredictable. The company is striving to make the boring things stay boring. Cotiviti calls it “The Labs Way” and here is how it does it:
- Treat everything as code. Everything. Infrastructure, data, everything. As an example, it performs pull requests to get status updates for executive reports.
- Test the code.
- Perform formal releases. Cotiviti builds its code in very specific ways, with extremely rigid gatekeeping processes.
- Releases produce immutable artifacts. Once something has been released, it will never change it. It could, but it doesn’t.
- Keep everything. Not in a huge, unorganized file system. In an organized repository.
- Failing tests mean a failed build.
- Design your systems to be automated. Cotiviti doesn’t do things that can’t be done automatically.
- Deal with those systems only through automation. This is probably the most challenging thing it does since it is extremely hard to convince developers to write their code to do this.
- Operations IS developing. Operations is not a bucket for development. Operations is a development task. You treat it like it is real code.
- Because everything is code. Everything.
- All defects are defects of code.
To achieve this, Cotiviti:
- Standardizes
- Evaluates
- Tests
- Discards the useless
- Owns it
None of this comes easy. It challenges norms. It pushes boundaries. It mandates discipline. (Pull quote – “Discipline is the bridge between goals and accomplishment.” – Jim Rohn) Your team has to understand you have a way to do things, there is a reason, and it will reduce risks and make sure everything works. It will keep the boring, boring.
You can watch Mykel’s full presentation from the All Day DevOps conference here (30 minutes). The other 56 presentations from the All Day DevOps Conference are available online, free-of-charge here.
Want more to read? This is blog series is reviewing sessions from the All Day DevOps conference from November which hosted over 13,500 registered attendees. Last week I discussed, “DevOps for Small Organizations: Lessons from Ed”. Next week, look for “Getting out of the Job Jungle with Jenkins,” a discussion by Damien Coraboeuf.