DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • Calendar View
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • Calendar View
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Cloud Native Now
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • CI/CD
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Sustainability
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • There’s No Value in Observability Bloat. Let’s Focus on the Essentials
  • Observability Leaders: The Mighty Minority Holding Their Own Against Outages
  • Cisco Acquires Splunk to Create Observability Powerhouse
  • Nobl9 Unfurls Reliability Center for Managing SLOs
  • Harness Launches Open Source Gitness Platform

Latest News Releases Hazelcast Demonstrates Cloud Efficiency, Real-Time Stream Processing of One Billion Events per Second

Hazelcast Demonstrates Cloud Efficiency, Real-Time Stream Processing of One Billion Events per Second

By: Deborah Schalm on March 17, 2021 Leave a Comment

Recent Posts By Deborah Schalm
  • Exabeam Reinvents Security Analytics with Fusion XDR and Fusion SIEM Cloud Products to Address Security Needs at Scale
  • New Study Reveals Importance of Optimized Strategy for the Selection, Support, and Maintenance of Open Source Software
  • Applitools Integrates With Rally for Fast and Automated Bug Management
More from Deborah Schalm
Related Posts
  • Hazelcast Demonstrates Cloud Efficiency, Real-Time Stream Processing of One Billion Events per Second
  • Hazelcast Strengthens Cloud-Native Offerings, Introduces New Managed Service
  • Hazelcast Cloud Enterprise Streamlines Application Modernization Projects Utilizing Multi-Cloud Architectures
    Related Categories
  • Latest News Releases
    Related Topics
  • Hazelcast
Show more
Show less

Benchmark achieved with 45 nodes enables extraordinary TCO and business insights from streaming data

SAN MATEO, Calif., March 17, 2021 /PRNewswire/ — Hazelcast, the fast cloud application platform, today announced that it successfully achieved a stream processing performance milestone of one billion events per second with 26-millisecond latency on 720 virtual CPUs (vCPUs) in a public cloud deployment. The Hazelcast In-Memory Computing Platform provides high speed, cloud-native, distributed stateful stream processing capabilities integrated with in-memory storage to power software applications with the highest throughput and lowest latency requirements. In addition to its performance, Hazelcast is architected to run with maximum efficiency to minimize the hardware footprint, delivering best-in-class total cost of ownership.

High-performance software systems are necessary today for more than traditional batch computations on large data sets which are common in any data-driven organization. These systems also provide the headroom in transaction processing and streaming analytics to perform more work in less time in a cost-effective deployment. This information gives businesses more freedom to experiment and identify new opportunities for competitive advantage.

AWS Builder Community Hub

The level of performance demonstrated in this benchmark is particularly valuable for improved machine learning (ML) training, as well as enhancing the decision making of an artificial intelligence (AI) powered application, such as fraud detection and other use cases that require automated, real-time decisions. Such algorithms become more complex and more resource-intensive in the search for greater accuracy, so cost-effective processing power is critical to enable ongoing evolution and improvement while keeping expenditures under control.

“I have been researching stream processing systems and open source developments for years, and I was very impressed to see these benchmark results,” said Asterios Katsifodimos, assistant professor at Delft University of Technology in Delft, Netherlands. “Until now, in the JVM-based stream processing world, a billion events per second requires painful parameterization, thousands of CPU cores, and will still only operate at multi-second latencies. Although most practical deployments require far smaller throughput than a billion events per second, this benchmark shows that Hazelcast Jet can handle massive data streams at millisecond latencies with a very small investment of cloud nodes, which leaves a lot of budget available for high-availability deployments.”

The benchmarking effort began with testing on small clusters of c5.4xlarge Amazon Web Services instances, each of which provides 16 vCPUs, using a data stream of one million events per second to identify the 99.99th percentile latency, representing a measurement that is one hundred times more strict than the usual practice of reporting at the 99th percentile. The test used a fast time resolution for updates of 20 milliseconds, which created a far more intensive workload than that of the 1-minute window specified in the NEXMark definition. Queries from the NEXMark benchmark suite, which simulate an online auction, were run on cluster sizes of 5, 10, 15 and 20 nodes to get the baseline latency at the specified percentile. In the 20-node cluster, query 5 of the NEXMark benchmark which asks, “Which auctions have achieved the highest price in the last period,” was the most complex with a 99.99th percentile latency of 16 milliseconds.

The benchmark then focused on finding the maximum throughput Hazelcast can offer at the same low latency as it scales from one node to as many as it needs to query one billion events per second. The update time resolution was relaxed to 500 milliseconds, which was still two orders of magnitude faster than the time window defined by the NEXMark benchmark. The latency percentile was also relaxed to the standard 99th percentile to keep the benchmark runs manageably short.

Query 5 was first run against a single node, where it achieved an impressive 25 million events per second. Repeating the test against increasing cluster sizes, Hazelcast exhibited nearly linear growth of throughput and demonstrated the minimum number of instances to achieve the targeted one billion events per second was 45 nodes, while still achieving a 99th percentile latency of 26 milliseconds.

“While one billion events per second may seem like overkill today, data is on the cusp of its next explosion with more applications moving to the cloud and eventually the edge, now accelerated by the rollout of 5G,” said John DesJardins, CTO of Hazelcast. “In combining the Hazelcast platform with the collaborative efforts with our partners, including IBM and Intel, customers can breathe easier knowing that their applications can process and compute large volumes of data in real-time, while keeping hardware costs under control.”

For more details on the benchmark and its results, visit the Hazelcast blog, “Billion Events Per Second with Millisecond Latency: Streaming Analytics at Giga-Scale.”

Additional Resources

  • Hazelcast Jet 4.4 is Released [Blog]
  • Hazelcast Jet Product Page [Web]
  • NEXMark Benchmark Information [Web]

About Hazelcast, Inc.

Hazelcast is the fast, cloud application platform trusted by Global 2000 enterprises to deliver ultra-low latency, stateful and data-intensive applications. Featuring real-time streaming and memory-first database technologies, the Hazelcast platform simplifies the development and deployment of distributed applications on-premises, at the edge or in the cloud as a fully managed service.

Hazelcast is headquartered in San Mateo, CA, with offices across the globe. To learn more about Hazelcast, visit https://hazelcast.com.

SOURCE Hazelcast, Inc.

Filed Under: Latest News Releases Tagged With: Hazelcast

« DevOps Institute Releases The Assessment of DevOps Capabilities to Help Practitioners, Teams and Organizations Advance DevOps Journeys
Hewlett Packard Enterprise fuels data-driven digital transformation with new products and partnerships for HPE Ezmeral Software »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Neo Quiz

Upcoming Webinars

Infosys Zero Cost Mainframe Transformations
Monday, September 25, 2023 - 11:00 am EDT
How PRINCE2 Improves Cybersecurity
Tuesday, September 26, 2023 - 11:00 am EDT
AWS and Sumo Logic: Observability With OpenTelemetry
Tuesday, September 26, 2023 - 1:00 pm EDT

GET THE TOP STORIES OF THE WEEK

Sponsored Content

JFrog’s swampUP 2023: Ready for Next 

September 1, 2023 | Natan Solomon

DevOps World: Time to Bring the Community Together Again

August 8, 2023 | Saskia Sawyerr

PlatformCon 2023: This Year’s Hottest Platform Engineering Event

May 30, 2023 | Karolina Junčytė

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Latest from DevOps.com

There’s No Value in Observability Bloat. Let’s Focus on the Essentials
September 22, 2023 | Tomer Levy
Observability Leaders: The Mighty Minority Holding Their Own Against Outages
September 22, 2023 | Jeff Stewart
Cisco Acquires Splunk to Create Observability Powerhouse
September 21, 2023 | Mike Vizard
Nobl9 Unfurls Reliability Center for Managing SLOs
September 21, 2023 | Mike Vizard
Harness Launches Open Source Gitness Platform
September 21, 2023 | Mike Vizard

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

Most Read on DevOps.com

Should You Measure Developer Productivity?
September 18, 2023 | Bill Doerrfeld
JFrog swampUP: Addressing the Advent of AI
September 18, 2023 | William Willis
Buildkite Acquires Packagecloud to Streamline DevOps Workflows
September 19, 2023 | Mike Vizard
What DevOps Teams Should Know About Phishing and the Supply Chain
September 19, 2023 | Gilad David Maayan
Oracle Previews Latest Java 21 Innovations
September 19, 2023 | Mike Vizard
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.