One of the big stumbling blocks naysayers claim DevOps can’t overcome is the perceived incompatibility of DevOps with high levels of compliance regulations. But forward-looking IT practitioners at financial institutions, healthcare organizations and government agencies alike have already started to show how untrue that perception really is.
The National Renewable Energy Laboratory (NREL) recently offered a perfect case study in this phenomenon. Not only do the key IT staffers in the trenches at NREL believe that DevOps is compatible with DevOps, but they actually used DevOps and continuous delivery principles to help the agency deliver compliance with a key piece of regulation around cloud integration within the NREL infrastructure.
The success came from a year-long project designed to garner the lab a needed update to its ‘Authority to Operate’ (ATO) designation under the Federal Risk and Authorization Management Program (FedRAMP) for the new integration of Amazon Web Services across its IT infrastructure. The project successfully doubled as a pilot program for DevOps and continuous monitoring, says Ryan Kelley, systems engineer at NREL. He explained that the agency wanted to take lessons learned from staffers in a very limited separate department who had previously dabbled with AWS and DevOps practices and apply them for a fresh approach in a broader project within NREL.
“The approach we took was, we’re not just going to try to jam all of our legacy toolsets into this and try to get them to work with the cloud,” he says. “We wanted to take a step back and say, ‘What’s the best approach for policy, what’s the best approach for toolsets and things like that for this new architecture?'”
Part of the impetus behind this fresh approach was the lingering memory of how painful it was for NREL to gain its ATO five years ago. At that time the lab used outside consultants to shepherd it through the process and it turned out to be “extremely painful, drawn out and very, very expensive,” Kelley says, explaining that the ATO governs
This time around, NREL hoped to find a way to use the expertise and resources it had in-house to streamline the process and not only comply with regulations but see a true return on its investment through a much improved infrastructure. According to Kelley, the project forced NREL to approach infrastructure almost like it was starting a new business.
“We had a fresh slate and we kind of had to build almost everything policy-wise and everything toolset-wise,” he says, explaining that in order to comply with the security requirements around achieving an ATO, NREL was most concerned about what’s called customer-responsible controls, which are often very configuration-centric. “That’s where having a centralized configuration management system pushes you towards compliance because it’s just – it gives us that central place to audit all of our configurations. Out of the gate, to get cloud underneath our authority to operate, we needed the ability to centrally manage and have our configurations approved.”
By approaching it as almost a new business, the IT team was able to make the cultural shifts necessary to achieve that through the improved teamwork and continuous delivery patterns that are the hallmark of DevOps.
“Enterprise IT tends to think that they have to deliver these big, huge projects all at once and tied up in a nice bow, whereas we’re trying to take the approach of getting a minimally viable product and then rapidly iterating on it,” he says, explaining that it took about a year to get the ATO, but that NREL is still iterating to improve the cloud integration overall.
Kelley says DevOps is providing a much needed shift for the organization due to the highly distributed nature of its programming resources. While the lab has a small cadre of IT staffers to run the backbone of IT infrastructure, the lab itself actually employs many programmers that actually work directly under what are essentially the lines of business for NREL, basically smaller sub-groups and departments working on individualized research projects. This ATO project showed that it is possible for the main IT group to interact with these smaller groups of developers to deliver quickly and effectively on projects and NREL plans to build on this success.
“You know, we’re dealing with the same problems everybody else is: complaints that we’re too slow to deliver, not responsive enough, and not working with the business enough,” he says, explaining that past pundits’ warnings for IT to ‘align itself to the business’ never before offered concrete advice on how to do that. “This was kind of one way that we did was just trying to find a project, develop a partnership and then sort of create a grassroots type of DevOps mentality. It definitely wasn’t top-down. And we’re continuing to work with these groups in this particular manner.”
Meanwhile, Kelley explains that the project also showed how continuous delivery toolsets can be instrumental in helping to with compliance demands.
“We feel that these new modern toolsets that let you treat your infrastructure as code actually gives our cybersecurity team more insight into what we’re doing because once you do that and start treating it like software, then it can be audited a lot easier,” he says. “It’s a lot easier than building each server on its own and then waiting for security to scan it and then waiting for them to get us a report back and then we deploy it.”
