DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • DevOps Chats
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Communities
    • AWS Community Hub
    • CloudBees
    • IT as Code
    • Rocket on DevOps.com
    • Traceable on DevOps.com
    • Quali on DevOps.com
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Digital Anarchist
  • Media Kit
  • About
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DevSecOps
  • DevOps Onramp
  • Practices
  • ROELBOB
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps

Home » News » Pepperdata® Code Analyzer for Apache Spark Highlights Performance Bottlenecks for Developers

Pepperdata® Code Analyzer for Apache Spark Highlights Performance Bottlenecks for Developers

By: Parker Yates on May 23, 2017 Leave a Comment

Recent Posts By Parker Yates
  • Appdome Announces Teams and Enhanced Workflow for DevOps Continuous Integration Models New Features Enable Group and Collaborative Mobile Integration Efforts and App Branding
  • Gigster Raises $20 Million Series B to Power the World’s Engineering
  • Infostretch Introduces New CloudBees Enterprise Jenkins Training Services, Driving DevOps Transformation
More from Parker Yates
Related Posts
  • Pepperdata® Code Analyzer for Apache Spark Highlights Performance Bottlenecks for Developers
  • Future of DevOps: Trends to Watch
  • CircleCI Report Finds DevOps Teams Adjusting to New COVID-19 Normal
    Related Categories
  • Latest News Releases
Show more
Show less

New Product Identifies Lines of Code and Stages that Cause Performance Issues Related to CPU, Memory, Garbage Collection, Network and Disk I/O 

CUPERTINO, Calif. – May 23, 2017 – Pepperdata, the DevOps for Big Data company, today announced Pepperdata Code Analyzer for Apache Spark, which provides Spark application developers the ability to identify performance issues and connect them to particular blocks of code within an application. Code Analyzer is a new product that follows on the heels of Pepperdata Application Profiler, which provides Hadoop and Spark developers with actionable recommendations for improving job performance.

“One of the most significant challenges in Big Data is achieving optimal performance,” said Ash Munshi, CEO of Pepperdata. “Code Analyzer fills a huge void in application development for Spark, helping developers optimize Spark applications for large-scale production. Developers are now empowered to improve the performance of Spark applications with new information and insight around the code, build, test and release phases.”

The performance metrics from Spark Web UI have historically been a challenge for developers to understand and contextualize, especially without having granular, time-series data on hand. Developers cannot easily drill down into and understand the problematic sections of an application that require optimization. Further, as Spark clusters typically run many applications in parallel, the Spark Web UI doesn’t inform developers how applications are impacted by other applications running on the cluster.

Pepperdata Code Analyzer allows Spark application developers to precisely measure how cluster resources – including CPU, memory, and network and disk I/O–are consumed by any particular block of application code. Code Analyzer delivers additional insight by combining application information from the Spark engine with granular time-series data for all applications running on a cluster. Dev teams are empowered with the ability to pinpoint the specific segment of their application code responsible for performance issues.

“I develop a lot of complex Spark code to perform ETL on Hadoop clusters. In these complex, large-scale systems, you must be able to understand where the performance bottlenecks are,” said Ian O’Connell, software engineer at Stripe and Pepperdata Technology Advisory Board member. “Pepperdata Code Analyzer for Apache Spark gives developers detailed time-series performance data for things like CPU, JVM memory and I/O usage overlaid against Spark job stages. I’m excited about the direction Pepperdata is moving — letting developers quickly see problems in time-series views and tie them back to their actual Spark application code will be a very useful tool for developers working on production Spark applications.”

Benefits of Code Analyzer include:

For Devs:

●      Identify which lines of code and which stages cause performance issues related to CPU, memory, garbage collection, network and disk I/O

●      Easily disambiguate resources used during parallel stages

●      Understand why run time variations occur for the same application

●      Determine whether performance issues are due to the application or other workloads on the cluster

For Ops:

●      Reduce the number of performance incidents in production

●      Easily communicate detailed performance issues back to developers

“Chartboost is the world’s largest mobile games-only advertising platform, reaching one billion active players around the world every month. Chartboost utilizes Apache Spark on large Amazon EC2 Hadoop clusters for machine learning and ET​L​ workflows,” said Michael McGowan, manager of Data Engineering at Chartboost. “Understanding Spark application performance in these complex environments is always a challenge. As a current use​r​ of Pepperdata Hadoop performance management tools, it has been great to work with Pepperdata on the development of Code Analyzer. It will give us comprehensive insight into Spark jobs.”

Pepperdata products and services are designed to accelerate the production use of Big Data applications by ensuring that performance is tightly integrated into the DevOps for Big Data cycle. Code Analyzer is integrated with Pepperdata products to provide an end-to-end DevOps solution, combining overall cluster awareness (monitoring, troubleshooting and alerting) with deep recommendations for improving the performance of individual jobs.

Availability and Pricing

Code Analyzer for Apache Spark will be available June 5 in early access, with general availability expected in Q3 2017. Pepperdata products are delivered to market as a combination of software running on customers’ clusters, on-premises or in the cloud, and as SaaS solutions. For pricing information or to schedule a demo, contact [email protected]. 

Helpful Links

●      Pepperdata website

●      Pepperdata Code Analyzer for Apache Spark

●      Pepperdata Application Profiler

●      Blog

●      Twitter

●      LinkedIn

Tweet This: [email protected] continues to empower developers with announcement of Code Analyzer for Apache #Spark http://ow.ly/1oxS30bT1sg #DevOps #BigData 

About Pepperdata

Pepperdata is the DevOps for Big Data company. Leading companies such as Comcast, Philips Wellcentive, and Zillow depend on Pepperdata to manage and improve the performance of Hadoop and Spark. Enterprise developers and operators use Pepperdata products and services to diagnose and solve performance problems in production and increase cluster utilization. The Pepperdata product suite improves communication of performance issues between Dev and Ops, shortens time to production, and increases cluster ROI. Pepperdata products and services work with customer Big Data systems both on-premise and in the cloud.

Founded in 2012, Pepperdata has raised $20M from investors including Citi Ventures, Signia Venture Partners and Wing Venture Capital, and attracted senior engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata is headquartered in Cupertino, California

— Parker Yates

Filed Under: Latest News Releases

Sponsored Content
Featured eBook
The Automated Enterprise

The Automated Enterprise

“The Automated Enterprise” e-book shows the important role IT automation plays in business today. Optimize resources and speed development with Red Hat® management solutions, powered by Red Hat Ansible® Automation. IT automation helps your business better serve your customers, so you can be successful as you: Optimize resources by automating ... Read More
« Feature Branching vs. Feature Flags: What’s the Right Tool for the Job?
Panaya Launches Release Dynamix, a Cloud-based Solution for ALM Enabling Rapid High Quality Software Delivery »

TechStrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

The ROI of Integration: Must-Have Capabilities to Maximize Efficiency and Communication
Thursday, August 18, 2022 - 11:00 am EDT
Best Practices For Writing Secure Terraform
Thursday, August 18, 2022 - 3:00 pm EDT
Transforming the Database: Critical Innovations for Performance at Scale
Tuesday, August 23, 2022 - 1:00 pm EDT

Latest from DevOps.com

Civo Report Surfaces Growing Cloud Lock-in Concerns
August 17, 2022 | Mike Vizard
Techstrong TV: Styra Declarative Authorization Service
August 17, 2022 | Alan Shimel
A Guide to Sustainable Application Modernization
August 17, 2022 | Bob Quillin
Overcoming Multi-Cloud Management Challenges
August 17, 2022 | Faiz Khan
Contrast Security Adds API Support to Security Platform
August 16, 2022 | Mike Vizard

GET THE TOP STORIES OF THE WEEK

Download Free eBook

The State of the CI/CD/ARA Market: Convergence
https://library.devops.com/the-state-of-the-ci/cd/ara-market

Most Read on DevOps.com

We Must Kill ‘Dinosaur’ JavaScript | Microsoft Open Sources ...
August 11, 2022 | Richi Jennings
What GitHub’s 2FA Mandate Means for Devs Everywhere
August 11, 2022 | Doug Kersten
Next-Level Tech: DevOps Meets CSOps
August 12, 2022 | Jonathan Rende
The Benefits of a Distributed Cloud
August 12, 2022 | Jonathan Seelig
Cycode Expands Scope of AppDev Security Platform
August 11, 2022 | Mike Vizard

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2022 ·Techstrong Group, Inc.All rights reserved.