OpenTelemetry is an open source project to provide a de facto standard trace API and a metric and log agent to end vendor lock-in
It’s no surprise that understanding the internal health of applications and systems is a high priority for developers and SREs. It’s a well-understood problem with dozens of vendors focused on helping companies diagnose application performance in real-time.
While the actual platform doing the metric, log or trace analysis tends to see most of the focus, a critical component of the data pipeline is, and has always been, the actual instrumentation. That includes APIs and SDKs for application traces as well as installable agents to gather metrics and logs. In the past, this has been a highly fragmented space with both open source and proprietary agents and SDKs. Most APM vendors have provided their own proprietary instrumentation libraries that, intentionally or not, create a vendor lock-in at a code-level that isn’t ideal for most organizations.
To address this challenge, an open source project was created with a highly aspirational goal of providing a de facto standard trace API along with a metric and log agent that would eliminate this fragmentation, create a unified standard and effectively end vendor lock-in.
The project, OpenTelemetry, was officially started in May 2019. It is a Cloud Native Computing Foundation sandbox project and came out of the merger of the OpenCensus and OpenTracing projects. They’ve since expanded their goal to include not only application tracing and code instrumentation, but also metric and log collection. They are essentially covering all the critical telemetry points in a system or application.
Imagine if, regardless of the platform you ultimately sent the data to, you could instrument your code with the same libraries, the same agents and a single set of binaries and libraries. It would dramatically simplify the deployment process, internal training and make it trivial to change or add analytics platforms as needed. This is the promise of the OpenTelemetry project.
A project like this doesn’t get far without significant support. This is where OpenTelemetry shows incredible promise to realize widespread adoption quickly. Nearly all the largest monitoring and cloud vendors including AWS, Google, Microsoft, Splunk, New Relic and many more have thrown their weight behind OpenTelemetry. They’re actively contributing and supporting the project. One by one, these organizations have come to the agreement that the telemetry component is not the differentiating feature. Instead, they are working together to provide a best-of-breed experience at the telemetry layer.
The project is in beta, and the potential is huge. OpenTelemetry modernizes every aspect of the telemetry system. It eliminates vendor lock-in. It will outperform the vast majority of existing agents. It simplifies installation and training by reducing dozens of agents and libraries to just one.
So where to get started?
It’s still early, but depending on your platform of choice, it may already support OpenTelemetry. If so, look into the existing libraries and test them out in your internal sandbox environments. You’re likely to find improvements over your existing telemetry tools, although there will certainly be gaps in things such as development language support and metric/log integrations. It should give you a sense of the scope, potential and maturity of the project and help to determine whether this is a viable solution for your next deployments.
The big question mark is whether this will live up to its potential. With the high-quality team assembled, the support of the community and industry and with a truly impactful mission, it will. Ideally, within a few years, this will be how we instrument and gather data from our applications regardless of where we send that data.