DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • DevOps Chats
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Communities
    • AWS Community Hub
    • CloudBees
    • IT as Code
    • Rocket on DevOps.com
    • Traceable on DevOps.com
    • Quali on DevOps.com
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Digital Anarchist
  • Media Kit
  • About
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DevSecOps
  • Leadership Suite
  • Practices
  • ROELBOB
  • Low-Code/No-Code
  • IT as Code
  • More Topics
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps

Home » Blogs » DevOps Practice » DevOps Challenges in C/C++ Projects

DevOps Challenges in C/C++ Projects

By: contributor on December 15, 2017 2 Comments

“C++ is not only the most powerful language in terms of performance, but it’s also a great language! It has true generics, it is expressive, and one of the most loved languages out there by programmers.”

Recent Posts By contributor
  • How to Ensure DevOps Success in a Distributed Network Environment
  • Dissecting the Role of QA Engineers and Developers in Functional Testing
  • DevOps Primer: Using Vagrant with AWS
More from contributor
Related Posts
  • DevOps Challenges in C/C++ Projects
  • Best of 2018: The Beauty of the COBOL Programming Language
  • Functional Safety in Embedded Design Starts with Code Development
    Related Categories
  • Blogs
  • DevOps Practice
    Related Topics
  • C/C++
  • developers
  • devops
  • programming language
Show more
Show less

— Diego Rodriguez-Losada, SW Engineer at JFrog and creator of Conan.io

This very powerful language that has unbeatable performance continues to be used and favored in all industries. However, while DevOps has become pervasive in other major domains of software development such as Java, implementing modern C and C++ DevOps hasn’t been easy, despite solutions out there that can actually help.

DevOps/Cloud-Native Live! Boston

Let’s take a look at the challenges C/C++ developers face, and then see how they can be solved.

tl;dr

C and C++ are are favored by many developers for many great reasons. However, implementing modern DevOps and continuous integration in C and C++ projects isn’t easy. It is still challenging due to the nature of the C/C++ programming languages and ecosystem. Today, we have the knowledge and experience to identify the requirements we need to meet today’s DevOps needs for C and C++. And I’ll demonstrate with the below case study that this is possible.

Building C and C++ Projects

Ideally, we want the CI/CD flow from the developer changes to production to be as smooth and fast as possible. However, when compared to other development languages, C/C++ developers face greater challenges:

  • Long build times: This is not because C and C++ compilers are slow. In fact, they are blazing fast and some of the most advanced and brilliant pieces of technology in the whole IT world. But, C/C++ have the problem of headers and the preprocessor basically copying every included file again and again in each translation unit, bloating the lines of code that they have to compile. There are some mechanisms as pre-compiled headers that can partly alleviate the issue for second and subsequent rebuilds. Also, C/C++ languages are used many times for huge projects compared with other languages, with millions lines of code (MLoc)—a typical AAA game has 20 MLoc.
  • Diverse platforms and ecosystems, different build systems: C and C++ are ubiquitous. They are used in computers, embedded devices, mobile devices and more. But in every platform they will use different compilers (such as MSVC cl.exe in Windows, GCC in Linux, apple-clang on OSX) and with many different build systems. CMake, MSBuild and Makefiles or autotools are widely used, but companies are also using many other build systems, even developing their own custom solutions.
  • Binary compatibility: C and C++ are built to native/binary code, but binaries compiled with a certain compiler version are not compatible with other compilers, or even with the same compiler of a different version. This problem is known as ABI (Application Binary Interface) incompatibility, which other languages don’t have to deal with. ABI incompatibilities can result in link errors, which are easy to catch at build time, but oftentimes can produce runtime errors.
  • Code inlining and embedding: Shared libraries directly embed native code of static libraries they are linking. But not only that, even static libraries might embed code declared in headers from another library. This means it might be necessary to rebuild binary artifacts from source code libraries that have not been modified at all, just because some of its dependencies have changed.
  • Lack of standard for OSS: OSS libraries can host their source code in GitHub, GitLab or Bitbucket, but that doesn’t make them ready to use. Users need to build binaries from sources (which can be painful), retrieve prebuilt binaries from a system package manager (which may be obsolete and in any case different for each platform) or by using other installers or precompiled modules (which must be downloaded from a website). After that, they typically also require different tweaks to the consumer build system, which is also different for each platform.

Modern DevOps Requirements for C and C++

Given the above list of challenges, what are the modern DevOps practices and tool requirements for C and C++ projects?

  • Manage and reuse binaries: Building from sources every time is very expensive, error-prone and doesn’t scale for large projects. Reusing binaries should be consistent for different platforms, and deal with the ABI compatibility issues.
  • Manage all binaries in a single repository or server: Creating and maintaining packages in several systems (such as apt in Debian, rpm in RH, nuget/choco in Windows and pkg in OSX) is very costly for developers or DevOps engineers. Being able to store, manage and retrieve binaries from the same location is a great simplifier of infrastructure and flows, thus a big time saver.
  • Adapt to any build system, platform and tool, from legacy to latest: Creating good CI and package management would be much simpler if everyone was using a single build system. But thinking projects can migrate to a build system is plainly naive. The cost of migrating some libraries or projects to a new build system is a huge investment that can take months. Processes and tools must be able to adapt to to deal with any build system and existing legacy tool.
  • Have good integrations with other tools: Not only build systems, but tools must integrate as simply as possible with other kinds of tools. For example, continuous integration platforms such as Travis and Appveyor cloud services for Linux/OSX and Windows, popular for being free for OSS projects, as well as Jenkins, which is the de facto standard in the industry for proprietary code. Also tools must integrate well with popular IDEs such as Visual Studio or XCode and testing, coverage, static analysis, multiple control version systems, etc.

Conan C/C++ Package Manager Case Study

Conan is a free OSS C/C++ package manager. It is decentralized, portable, and multi-platform, running on many operating systems, such as Windows, Linux, Mac, BSD and Solaris, but targeting any possible target of C/C++ applications.

Here’s how Conan addresses the above requirements:

  • Conan is able to build, upload, and share binaries for any number of different configurations. It manages binaries, retrieving the correct one for each corresponding platform as needed, and is also able to build from sources when necessary, minimizing ABI incompatibility issues.
  • Conan packages and pre-built binaries can be uploaded to a server and shared within They can be shared privately within a team or companywide, using the OSS conan_server or JFrog Artifactory Pro. If desired, Conan packages can be distributed worldwide using JFrog Bintray either with private or public repositories, completely free for OSS projects. All different binaries for all supported platforms can be uploaded and managed in the same server with exactly the same interface.
  • Conan works with any build system. It includes built-in integrations with CMake, Visual Studio, Makefiles, SCons, QMake and more. It also has good user-side extensibility to support any other build system, using “generators.” This means extensions can be developed to translate the Conan dependencies model to any build system format and generate files that can be read by those build systems.
  • Conan integrates well with different CI systems, with specific tools and helpers for Travis, Appveyor and GitLab, and also features custom DSLs in the Jenkins Artifactory Plug-in.

Conan package recipes are written in Python, so they can be used to easily customize behavior to a great extent with standard Python syntax, allowing users to easily plug-in many different tools that were previously mentioned in this article: static analysis, testing, code coverage, and more.

Let’s take a look at a typical flow, from a DevOps perspective, to illustrate how Conan works. First, a developer makes some source code changes to a specific library (LibB), committing and pushing these changes to the shared version control system server (Git Repo).

Diagram: CI Cycle example for C/C++ with Conan, Artifactory and Jenkins (build systems: CMake, VS, MakeFile)

Then the CI system will listen to the Git repository, ideally with Git hooks since polling is not recommended, and fire a job for it (Jenkins). Below is an example of how this would be defined with a Jenkins pipeline:

The first stage in the above example is straightforward. The second stage invokes the package creation that is internally split into the two subtasks:

  1. The LibB dependencies are retrieved in binary form, to avoid rebuilding them from sources and wasting time.
  2. The dependency graph for all transitive dependencies is built and one or more files are written. In this example, LibB uses CMake as the build system and Conan generates a “conanbuildinfo.cmake” file that includes all of the dependencies information to enable LibB find and correctly link to them. Conan will then call the LibB build system to generate a binary package for it. And finally, in the last Jenkinsfile stage “Upload packages,” the generated binary is uploaded to the Conan package server (JFrog Artifactory). The LibB binary package is immediately available for consumption by developers or other CI jobs.

This example illustrates a build configuration for a 32-bit architecture as specified in the  “conan create” command with the “-s arch=x86” argument. Note that many additional configurations can be built in parallel using parallel tasks and jobs for different compilers, OS, architectures and more.

Conclusion

C and C++ have extra challenges compared to other programming languages in development. This also makes it more challenging when trying to implement modern DevOps, but not impossible. C/C++ has been overlooked and it’s time we update the tools to meet the needs of developers. What DevOps principles and practices are you implementing with your C/C++ projects?

About the Author / Baruch Sadogursky

Baruch Sadogursky is the Developer Advocate at JFrog. For a living he hangs out with JFrog’s tech leaders, writes code around the JFrog platform and its ecosystem, and then speaks and blogs about it all. He has been doing this for the last dozen years or so, and enjoys every minute of it. He also is a professional conference speaker on DevOps, Java and Groovy topics, and is a regular at the industry’s most prestigious events including JavaOne (where he was awarded a Rock Star award), DockerCon, Devoxx, DevOps Days, OSCON, Qcon and many others.  His full speaker history is available on Lanyrd. Follow him on Twitter @jbaruch on Twitter, or read his blogs here and here.

Filed Under: Blogs, DevOps Practice Tagged With: C/C++, developers, devops, programming language

Sponsored Content
Featured eBook
The Automated Enterprise

The Automated Enterprise

“The Automated Enterprise” e-book shows the important role IT automation plays in business today. Optimize resources and speed development with Red Hat® management solutions, powered by Red Hat Ansible® Automation. IT automation helps your business better serve your customers, so you can be successful as you: Optimize resources by automating ... Read More
« What Developers Want, and How to Give It to Them
Tactical Planning »

TechStrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

LIVE WORKSHOP - Boost Your Serverless Application Availability With AIOps on AWS
Wednesday, May 25, 2022 - 8:00 am EDT
Supercharge Your AWS Cloud Platform With Self-Service Cloud Ops
Thursday, May 26, 2022 - 1:00 pm EDT
Abracadabra: Achieving Zero Downtime With ANY Observability Tool
Tuesday, May 31, 2022 - 11:00 am EDT

Latest from DevOps.com

Could Buying VMware Bring Broadcom Hybrid Cloud Bona Fides?
May 24, 2022 | Dan Kirsch
Competing Priorities Prevent Devs From Creating Secure Code
May 24, 2022 | Pieter Danhieux
DevOps/Cloud-Native Live Boston: Get Certified, Network and Grow Your Career
May 23, 2022 | Veronica Haggar
GitLab Gets an Overhaul
May 23, 2022 | George V. Hulme
DevOps and Hybrid Cloud: Life in the Fast Lane?
May 23, 2022 | Benjamin Brial

Get The Top Stories of the Week

  • View DevOps.com Privacy Policy
  • This field is for validation purposes and should be left unchanged.

Download Free eBook

The 101 of Continuous Software Delivery
New call-to-action

Most Read on DevOps.com

DevOps Institute Releases Upskilling IT 2022 Report 
May 18, 2022 | Natan Solomon
DevSecOps Deluge: Choosing the Right Tools
May 20, 2022 | Gary Robinson
Creating Automated GitHub Bots in Go
May 18, 2022 | Sebastian Spaink
Managing Hardcoded Secrets to Shrink Your Attack Surface 
May 20, 2022 | John Morton
Is Your Future in SaaS? Yes, Except …
May 18, 2022 | Don Macvittie

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2022 ·Techstrong Group, Inc.All rights reserved.