DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB

Home » Blogs » Enterprise DevOps » Top Challenges When Scaling DevOps in the Enterprise and How to Conquer Them

Top Challenges When Scaling DevOps in the Enterprise and How to Conquer Them

Avatar photoBy: Rami Sass on January 30, 2018 Leave a Comment

Preparing to adopt DevOps can be a challenge for organizations of any size. Scaling DevOps and bringing it to an enterprise demands a special strategy. Use these tips to build yours.

Related Posts
  • Top Challenges When Scaling DevOps in the Enterprise and How to Conquer Them
  • Webinar: Scaling DevOps for the Enterprise with GitHub
  • Webinar: Adopting DevOps @ Scale: Lessons learned at Hertz, Kaiser Permanente and lBM
    Related Categories
  • Blogs
  • Enterprise DevOps
    Related Topics
  • DevOps environment
  • enterprise
  • scaling challenges
Show more
Show less

DevOps principles apply to all organizations. Everybody has business goals that can be met and exceeded by following DevOps practices. However, an organization’s size will inevitably impact the level of difficulty it will face during implementing and scaling DevOps. A large enterprise organization will encounter more daunting obstacles than a smaller organization, even if their problems are similar in nature.

TechStrong Con 2023Sponsorships Available

Regardless of your organization’s size, you are probably wondering what to expect during your DevOps transformation and asking yourself which strategies you can use to overcome any complications. Luckily, you now have “The DevOps Handbook” to turn to for advice. This book is an excellent resource that distills the most important insights learned from difficult experiences into straightforward steps and solutions for overcoming common DevOps hurdles.

DevOps organizations move much faster than many other organizations today without sacrificing quality, performance or security. This blog post will cover some helpful strategies from “The DevOps Handbook” that will help you overcome challenges your organization might face during its shift to DevOps.

Changing Your Organization’s Culture

The first hurdle in the DevOps process is managing culture and expectations. Scaling DevOps throughout the organization requires changing how your business views itself. It also involves rethinking your process for delivering value to the customer. The goal is to get the changes you make in your products and services to the customer faster. In technical terms, this boils down to how often deployments happen. The DevOps process uses positive feedback loops to deliver value to the customer quickly.


Related Content:

What is an Agile IT Organization?

Everything Ops: Are DevOps Principles Being Applied Too Broadly?


To achieve this goal, “The DevOps Handbook” says that it is necessary to shift your focus from cost-optimization to speed-optimization. Optimizing speed does not require a massive, top-down reorganization. Rather, it means giving teams more autonomy. Different skills (such as development, operations and information security) should be embedded into existing teams. This will make teams become self-serving, which will enable them to develop, deploy and operate their software more safely and effectively. Majorly successful organizations such as Netflix and Amazon attribute their success to this approach.

However, building self-serving teams is just one part of the culture change. You also must change how your people think about themselves and their work. Let’s take Toyota, the pioneers of the lean manufacturing process, as an example. The “Toyota Way” has propelled the company’s success, as Mike Rother wrote about in “Toyota Kata.” In the book, he explains that while structural reorganization is an important step toward overall improvement, it is also critical to invest in the betterment of your employees. You have to build upon their technical work abilities and develop the habits of continual learning and experimentation. Toyota accomplished this by using what Rother called The Improvement Kata, which is a model for achieving goals that is based on coordinating theory and evidence.

DevOps organizations integrate continual learning and experimentation into their day-to-day work and decision-making. Verifiable experimentation, such as A/B tests and cohort analyses, advance business goals. The challenge is building the data-driven learning framework. Product owners and other decision makers need a variety of business telemetry. Accordingly, engineers must collaborate and implement whatever is necessary to provide it. Everyone can—and should—use this telemetry to understand what is happening with the business and IT at any point in time.

Etsy is a good example of an organization that has implemented telemetry well. According to “The DevOps Handbook,” in 2011, Etsy started tracking 200,000 business-layer metrics, displaying 30 of the most important metrics on its deployment dashboard. By 2014, the company had tracked more than 800,000 of these business metrics. This high level of dedication to telemetry made its engineers’ jobs easier with quick business and technical documentation, especially after deployments.

The next hurdle is combining telemetry with the practice of continuous learning and experimentation. The two practices work best together when the process is moving as quickly as possible.

Faster Iterations

Addressing speed-related challenges requires more of a technical effort than the others. DevOps requires fast feedback loops at all levels, which begins and ends with a new set of engineering practices.

Trunk-Based Development

First, you should start using Trunk-Based Development (TBD). TBD is a model for source-control branching, in which multiple developers work together on code from one, individual branch. With TBD, developers check in their code to trunk one or more times daily. Using this method has powerful knock-on effects. Automated tests run against the entire system, so it is immediately clear if something is broken. TBD also reduces long-running feature branches that may eventually create difficult merge conflicts or introduce unexpected bugs.

Another great thing about TBD is that it naturally keeps changes smaller, which means that they are easier to review, test and reason about, all while keeping your code in a releasable state. It can also help you reduce your total number of environments, which can easily save tens of thousands of dollars every month.

Automated Deployment

Next, you should automate your deployment process. This requires automating as many of the intermediate steps as possible, including developer workstation setup, test environment setup, environment promotion, the deployment itself and telemetry collection. This is a lofty goal, but taking it to the extreme creates powerful business outcomes. More people are able to deploy more often. Deployments become smaller, safer and more routine, and the number of production bugs is reduced.

Making this happen is no small task since everybody involved must adopt the “automation first” mindset. Start by automating the more error-prone and repetitive tasks. The effort will be worth your time, since automation increases throughput and reduces errors.

Etsy is a big fan of automated deployment. According to an Etsy case study from 2014 that can be found in “The DevOps Handbook,” “deployments are performed by anyone who wants to perform a deployment, such as Development, Operations, or Infosec. The deployment process at Etsy has become so safe and routine that new engineers will perform a production deployment on their first day at work — as have Etsy board members and even dogs!” You may not want actual dogs in your control room, but you certainly want that level of confidence and satisfaction.

Integrating Information Security

This amount of automation sounds fantastic for developers and operations, but what about information security? For them, it’s not necessarily true that topics such as compliance and security are treated as afterthoughts. DevOps is successful because it uses quick feedback loops to identify and fix problems early on in the software development process, but information security and compliance usually come into the picture at the very end, creating huge blockages and problems for everyone involved.

Information security is another skill such as QA or operations, so it can be embedded in cross-functional teams. Then, teams can work together to address known security issues, such as testing for package vulnerabilities as part of CI. Testing for proper secrets, such as a database connection string, can be handled with automation, too. Teams can also integrate static analysis to detect possible problems before the code is executed.

Continuous delivery practices also apply here. Security-related metrics such as total password resets, potential SQL injections, XSS attacks, core dumps and other strange events should be integrated into the deployment pipeline. Doing so provides visibility and immediately actionable information to the entire team during incidents.

Bottom-Up Scaling

Large enterprise organizations are structurally complicated. There are many leaders and smaller teams, each with their own mission and culture. As such, trying to change things top-down is a big mistake—it simply does not work. Scaling out from the bottom-up is a much better approach, and there are a few ways to accomplish this.

Starting small is the best advice. We recommend that you find a small group of like-minded individuals and give them technical autonomy, goals and your trust to improve the business. This will likely help them tackle technical debt issues that have hindered them in the past. They’ll try new approaches and learn down in the trenches.

Unfortunately, there is no such thing as DevOps boot camp—only principles and technical practices. It’s up to everyone in the team (and the organization in general) to figure out what DevOps means to them. Hopefully, their results will speak for themselves. Naturally, other teams will become interested in how they work and think. In time, some team members may even be ready to train their peers.

Ross Clanton at Target uses a “technological innovation center,” which is more commonly known as a DevOps Dojo. The Dojo coaches help teams across Target’s IT department elevate their states of practice. The “30-Day Challenge” is their most intensive training. Blocked development teams bring their work with them to work alongside dedicated coaches and engineers. The goal of the challenge is to unblock your team in less than 30 days. According to Clanton, taking part in this challenge has helped many teams finish work that would normally take months in just a matter of days. Completing these challenges has solved many internal problems, and has ultimately made the organization as a whole more productive.

Conclusion

There is more than one way to scale DevOps in your organization. In this blog post, we have reviewed three of the main DevOps principles that will help you along your journey. First, you need to begin changing your organization’s culture and organizational structure to allow for a “divide and conquer” strategy. Next, establish protocols that will help you find errors early. Doing so will increase the speed of feedback loops, which will make for quicker delivery of customer value. Third, invest in continuous learning and experimentation of new ideas.

Using this and other advice from “The DevOps Handbook,” you can work through your DevOps transformation with confidence. When in doubt, we suggest that you consult this excellent resource for more helpful tips.

— Rami Sass

Filed Under: Blogs, Enterprise DevOps Tagged With: DevOps environment, enterprise, scaling challenges

« Before the Singularity
Why DevOps Principles Fit Well in Highly Regulated Industries »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Evolution of Transactional Databases
Monday, January 30, 2023 - 3:00 pm EST
Moving Beyond SBOMs to Secure the Software Supply Chain
Tuesday, January 31, 2023 - 11:00 am EST
Achieving Complete Visibility in IT Operations, Analytics, and Security
Wednesday, February 1, 2023 - 11:00 am EST

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Stream Big, Think Bigger: Analyze Streaming Data at Scale
January 27, 2023 | Julia Brouillette
What’s Ahead for the Future of Data Streaming?
January 27, 2023 | Danica Fine
The Strategic Product Backlog: Lead, Follow, Watch and Explore
January 26, 2023 | Chad Sands
Atlassian Extends Automation Framework’s Reach
January 26, 2023 | Mike Vizard
Software Supply Chain Security Debt is Increasing: Here’s How To Pay It Off
January 26, 2023 | Bill Doerrfeld

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

What DevOps Needs to Know About ChatGPT
January 24, 2023 | John Willis
Microsoft Outage Outrage: Was it BGP or DNS?
January 25, 2023 | Richi Jennings
Five Great DevOps Job Opportunities
January 23, 2023 | Mike Vizard
Optimizing Cloud Costs for DevOps With AI-Assisted Orchestra...
January 24, 2023 | Marc Hornbeek
A DevSecOps Process for Node.js Projects
January 23, 2023 | Gilad David Maayan
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.