Latest DORA Report From Google Surfaces Raft of DevOps Challenges

The annual Accelerate State of DevOps 2022 report published today by the DevOps Research and Assessment (DORA) team at Google found the percentage of high performers is at a four-year low, with the percentage of low performers rising dramatically from 7% in 2021 to 19% in 2022.

Based on a survey of 1,350 DevOps professionals, the report evaluated organizations based on the five DORA metrics: Deployment frequency, lead time for changes, time to restore service, change failure rate and operational performance. The DORA team also identified clusters of DevOps teams ranging from “starters” to “flowing” based on their overall maturity. Only 17% of respondents achieved a ‘flow’ state defined by high reliability, high stability and high throughput characteristics.

The DORA survey sample shifted this year to include more respondents that are earlier in their careers than in previous reports. However, Claire Peters, DORA research lead at Google, said that while there are a lot of factors that impact those metrics, the current hypothesis is that the COVID-19 pandemic has taken a toll on DevOps productivity as more DevOps professionals continue to work remotely.

The report noted a marked shift in deployment targets requiring new skills, with 54% of respondents now working with container artifacts.

Overall, the report finds software delivery performance is beneficial to organizational performance when operational performance is also high. The challenge is the number of organizations that have achieved high operational performance is relatively small. In fact, the report also noted that site reliability engineering (SRE) practices had a negative impact on software delivery performance until an organization achieved a high level of SRE maturity.

The report also concluded that organizations that built software on and for the cloud tended to have 1.4 times higher organizational performance than those that didn’t. The percentage of respondents using public clouds is 76%, up from 56% in 2021, while the number of respondents not using a cloud stands at 10.5%. Usage of multiple public clouds is at 26%, while 35% reported using a private cloud.

Less clear is the impact on DevOps teams of shifting application security left. The report suggested the biggest challenge organizations face when it comes to achieving application security have more to do with cultural issues than any technical shortcomings. The report notes that organizations that focus on accelerating software delivery without implementing meaningful DevSecOps best practices find themselves in a vicious counterproductive cycle that results in applications being successfully deployed in production environments less often.

Peters noted that there is greater developer fatigue because organizations attempt to address security issues either just before an application is deployed or they remediate applications after they are deployed. Organizations that have low levels of security practices are 1.4 times more likely to experience developer burnout.

The DORA team used both the supply-chain levels for secure artifacts (SLSA) framework defined by Google and the Secure Software Development Framework (SSDF) defined by the National Institute of Standards (NIST) to evaluate organizations. The most widely-adopted practice is application security scanning within a continuous integration/continuous delivery (CI/CD) system, with 63% of respondents reporting these tools are “very” or “completely” established. Preserving code history and using build scripts are also highly established, the report finds.

In general, Peters said the report suggested the biggest predictor of application security success was whether an organization had a high-trust, low-blame culture focused on performance. Organizations that have these cultures tend to have higher organizational performance. Similarly, organizations with teams that felt supported through funding and leadership sponsorships tended to have higher organizational performance, the report noted. In addition, team stability and positive perceptions about one’s team also tended to lead to higher levels of organizational performance. Lastly, companies that offer flexible work arrangements tended to see higher levels of organizational performance.

Each organization, as always, will implement DevOps best practices in ways that best fit its internal culture. However, the latest DORA report makes it clear that the more empowered a DevOps team is the more proficient they tend to become.