Features

RapidAPI Adds Library to Invoke Kafka via API

RapidAPI announced it has added support for the open source Kafka messaging software as another type of application programming interface (API) that DevOps teams can centrally access.

An open source library provided by RapidAPI, available in beta, makes it possible to discover Kafka clusters and topics, view schemas and test and consume records from a browser via either the RapidAPI Marketplace or the RapidAPI Enterprise Hub. The library is based on Apache Avro and Confluent wire formats found in the open source confluent-schema-registry library. RapidAPI extended that library to add support for JSON and a protocol buffer, which is now being shared with the open source Kafka community.

RapidAPI CEO Iddo Gino said DevOps teams can now discover all available Kafka clusters and instances in the same location where they find, test and connect to microservices based on REST, SOAP and GraphQL APIs. That effort is designed to complement initiatives, such as the AsyncAPI specification, that is now being advanced under the auspices of the Linux Foundation, Gino noted.

RapidAPI is adding this capability because usage of Kafka to enable asynchronous communication across event-driven applications has significantly increased, Gino said. As the number of digital business transformation initiatives increased in the wake of the COVID-19 pandemic, Gino said reliance on asynchronous messaging platforms such as Kafka became a requirement. In many cases, synchronous communications across highly distributed applications simply isn’t feasible, Gino added.

That doesn’t mean asynchronous requirements will replace the need for traditional synchronous APIs. Rather, developers will find themselves incorporating a greater mix of both as enterprise IT environments become more distributed, noted Gino. Applications are increasingly become one large puzzle that developers need to construct using various classes of APIs, added Gino.

The library created by RapidAPI is expected to be generally available in the third quarter. In the meantime, adoption of Kafka continues to accelerate. The open source community that oversees development of Kafka, under the auspices of the Apache Software Foundation, claims more than 80% of all Fortune 100 companies are using the platform for everything from constructing data pipelines to running streaming analytics applications. In some cases, data is being queried directly while in transit versus waiting for it to be stored on-premises or in the cloud.

Regardless of how Kafka is employed, developers are rediscovering event-driven architectures, that can trace their lineage to the 1980s, for building applications. The difference now is there an open source framework for handling the asynchronous communications required to build and deploy those applications at scale.

It’s not quite clear how the rise of event-driven applications will impact DevOps workflows, as event-driven applications become more commonly built and deployed. However, as the APIs become more widely employed within the context of these applications, it should become simpler to incorporate them.

The challenge is event-driven architectures typically requires developers to think differently about how their applications are structured, so it might be a while before Kafka – and other similar platforms – are pervasively employed across the enterprise.

Mike Vizard

Mike Vizard is a seasoned IT journalist with over 25 years of experience. He also contributed to IT Business Edge, Channel Insider, Baseline and a variety of other IT titles. Previously, Vizard was the editorial director for Ziff-Davis Enterprise as well as Editor-in-Chief for CRN and InfoWorld.

Recent Posts

Building an Open Source Observability Platform

By investing in open source frameworks and LGTM tools, SRE teams can effectively monitor their apps and gain insights into…

13 hours ago

To Devin or Not to Devin?

Cognition Labs' Devin is creating a lot of buzz in the industry, but John Willis urges organizations to proceed with…

14 hours ago

Survey Surfaces Substantial Platform Engineering Gains

While most app developers work for organizations that have platform teams, there isn't much consistency regarding where that team reports.

1 day ago

EP 43: DevOps Building Blocks Part 6 – Day 2 DevOps, Operations and SRE

Day Two DevOps is a phase in the SDLC that focuses on enhancing, optimizing and continuously improving the software development…

1 day ago

Survey Surfaces Lack of Significant Observability Progress

A global survey of 500 IT professionals suggests organizations are not making a lot of progress in their ability to…

1 day ago

EP 42: DevOps Building Blocks Part 5: Flow, Bottlenecks and Continuous Improvement

In part five of this series, hosts Alan Shimel and Mitch Ashley are joined by Bryan Cole (Tricentis), Ixchel Ruiz…

1 day ago