DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Cisco Bets on OpenTelemetry to Advance Observability
  • 5 Technologies Powering Cloud Optimization
  • Platform Engineering: Creating a Paved Path to Reduce Developer Toil
  • Where Does Observability Stand Today, and Where is it Going Next?
  • Five Great DevOps Job Opportunities

Home » Latest News Releases » Varada Open-Sources Its Workload Analyzer to Help Data Teams Optimize Data Lake Queries

Varada Open-Sources Its Workload Analyzer to Help Data Teams Optimize Data Lake Queries

By: Deborah Schalm on February 2, 2021 1 Comment

Workload Analyzer gives data engineers holistic visibility into performance of Presto®  clusters, enabling resource optimization and improved service to business-wide users of Big Data analytics
Recent Posts By Deborah Schalm
  • Exabeam Reinvents Security Analytics with Fusion XDR and Fusion SIEM Cloud Products to Address Security Needs at Scale
  • New Study Reveals Importance of Optimized Strategy for the Selection, Support, and Maintenance of Open Source Software
  • Applitools Integrates With Rally for Fast and Automated Bug Management
More from Deborah Schalm
Related Posts
  • Varada Open-Sources Its Workload Analyzer to Help Data Teams Optimize Data Lake Queries
  • Accurics Adds Compliance Control Support to Code Analyzer
  • Keysight Enables Customers to Test Performance of Millimeter-wave Innovations in 5G, Aerospace/Defense and Satellite Communications
    Related Categories
  • Latest News Releases
    Related Topics
  • Varada
Show more
Show less

 

TEL AVIV, Israel — February 2, 2021 — Varada, the data lake query acceleration innovator, today announced that it has open-sourced its Workload Analyzer for Presto, including both Trino (formerly known as PrestoSQL) and PrestoDB, making the source code available to everyone via Github. The Workload Analyzer is a free, easy-to-use tool that offers visibility into how Big Data and analytics workloads are performing, offering users insights into how to improve performance and optimize resources. Download the Workload Analyzer here.

TechStrong Con 2023Sponsorships Available

“Presto democratized Big Data, exponentially expanding the number of business users that can ask questions to a Big Data infrastructure and enlarging the number of underlying data sources they can query,” said Ori Reshef, vice president of products at Varada. “But as the number of users within an organization grows, the challenge of DataOps teams is to keep queries running quickly, delivering results in a timely way so that those users can do their jobs. Unfortunately, DataOps teams are only able to get bits and pieces of the information they need to optimize resources from Presto itself. So Varada built the Workload Analyzer to give DataOps teams deep and actionable insights.”

 

The Workload Analyzer collects details and metrics on every query, aggregates and extracts information, and delivers dozens of charts describing all the facets of cluster performance. For the first time, data engineers have a holistic view of their cluster and can drill down into pain points to determine what queries to optimize and how. Download a sample Presto Workload analysis report.

 

The Workload Analyzer is compatible with PrestoDB and Trino. The Workload Analyzer script runs safely within the Presto cluster in the user’s Virtual Private Cloud (VPC), collecting and analyzing query statistics (JSONs). No data leaves the cluster and the tool does not require any external resources. The Workload Analyzer has already been tested on dozens of massive scale production clusters, resulting in zero impact on query performance.

 

Using the Workload Analyzer, data teams can:
  • Learn how resources are used on an hourly and weekly basis and define scaling rules

  • Identify heavy spenders and improve the pipeline
  • Improve predicate pushdown and significantly reduce IO and CPU
  • Identify “hottest” data
  • Improve JOINs performance
  • Provide a better production roll-out experience and identify upgrade risks upfront

 

“We’re already seeing this tool used in amazing ways,” said Reshef. “For example, one company is using it as a quality assurance tool for daily tests on large clusters. Another is using it for strategic planning to understand the best data sets to query for business users, while allocating resources effectively to significantly reduce costs. The number of use cases continues to rise.”

 

Presto: A Tool of Choice for Data-driven Companies
Presto is an open source distributed SQL query engine for running interactive analytic queries. Presto offers many benefits, most notably its ability to quickly run queries on a wide variety of data sources all at once, including ‘raw,’ unmodeled data. With this capability, as well as other unique advantages, Presto has quickly become a tool of choice for many significant data-driven companies.

 

The Varada Commitment to the Trino and PrestoDB Communities
“As part of our deep commitment to the PrestoDB and Trino communities, Varada decided to release a standalone, open source version of our Workload Analyzer tool so that any Presto user can evaluate potential performance improvements in their cluster,” said Eran Vanounou, CEO of Varada. “The tool will help PrestoDB and Trino users optimize their clusters on their own using their existing solutions. Of course, we anticipate that after discovering the existing inefficiencies within their clusters, many users will want to further evaluate how adding an indexing layer to PrestoDB or Trino can help them vastly improve performance. We will be more than happy to demonstrate how the Varada Data Platform can do just that.”

 

Varada leverages Presto in its innovative query acceleration engine, the Varada Data Platform. A big data infrastructure solution for fast analytics on thousands of dimensions, the Varada Data Platform became generally available in December 2020. Varada’s proprietary indexing layer runs on top of Presto, improving Presto’s query response time by x10-x100.

 

About Varada
The Varada mission is to enable data practitioners to go beyond the traditional limitations imposed by data infrastructure and instead zero in on the data and answers they need—with complete control over performance, cost and flexibility. In Varada’s world of big data, every query can find its optimal plan, with no prior preparation and no bottlenecks, providing consistent performance at a petabyte scale. Varada was founded by veterans of the Dell EMC XtremIO core team, and is dedicated to leveraging the data lake architecture to take on the challenge of data and business agility. Varada has been recognized in the Cool Vendors in Data Management report by Gartner, Inc. For more information, visit: https://varada.io/

 

Presto® is a trademark of The Linux Foundation.

Filed Under: Latest News Releases Tagged With: Varada

« Delphix Accelerates YoY Growth Rate by Over 85%, Swings to Profitability
Pattern89 Forecasts Social Media Ad Trends for Super Bowl, COVID-19 Visuals in New Report »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Shipping Applications Faster With Kubernetes: Myth or Reality?
Wednesday, February 8, 2023 - 1:00 pm EST
Why Current Approaches To "Shift-Left" Are A DevOps Antipattern
Thursday, February 9, 2023 - 1:00 pm EST
Log Love: Monitoring, Troubleshooting, Forensics and Biz Analytics
Tuesday, February 14, 2023 - 11:00 am EST

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Cisco Bets on OpenTelemetry to Advance Observability
February 7, 2023 | Mike Vizard
5 Technologies Powering Cloud Optimization
February 7, 2023 | Gilad David Maayan
Platform Engineering: Creating a Paved Path to Reduce Developer Toil
February 7, 2023 | Daniel Bryant
Where Does Observability Stand Today, and Where is it Going Next?
February 6, 2023 | Tomer Levy
Five Great DevOps Job Opportunities
February 6, 2023 | Mike Vizard

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Automation Challenges Holding DevOps Back
February 1, 2023 | Mike Vizard
Three Trends That Will Transform DevOps in 2023
February 2, 2023 | Dan Belcher
Red Hat Brings Ansible Automation to Google Cloud
February 2, 2023 | Mike Vizard
The Ultimate Guide to Hiring a DevOps Engineer
February 2, 2023 | Vikas Agarwal
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.