DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Postman Releases Tool for Building Apps Using APIs
  • What DevOps Leadership Should Look Like
  • Things We Should Acknowledge, Part One: Hiring Sucks
  • HPE to Acquire OpsRamp to Gain AIOps Platform
  • Oracle Makes Java 20 Platform Generally Available

Home » Blogs » Continuous Delivery » Kubernetes Jenkins Master-Slave: Scaling the Scalability Issue

Kubernetes Jenkins Master-Slave: Scaling the Scalability Issue

Avatar photoBy: Lucky Kumar Sappa on July 19, 2018 1 Comment

LinkedIn, Sony, Dell and many other companies use Jenkins as a CI/CD tool and builds happen with substantial code bases that run 365 days in parallel. Generally, you may not have as much code to build, but the slave that’s created during the build process is in an upstate even after the build is completed. That can result in higher cost, higher unnecessary resource utilization and a more complex delivery pipeline.

Related Posts
  • Kubernetes Jenkins Master-Slave: Scaling the Scalability Issue
  • Webinar: Scaling Jenkins with Docker and Kubernetes
  • CLOUDBEES ANNOUNCES INDUSTRY’S FIRST CERTIFICATION PROGRAM FOR JENKINS ENGINEERS
    Related Categories
  • Blogs
  • Continuous Delivery
  • DevOps Practice
Show more
Show less

Is there the process to overcome this situation? Yes. The solution is scalability.

Jenkins Scalability

One of the strongest features that Jenkins has is scaling—pushing beyond its normal limits of productivity. In this article we’re taking a look at the Jenkins master/slave model with one central Jenkins instance, referred to as master, and couple of slave executables called slaves, with the master responsible for scheduling the jobs across the slaves. Some features of slaves are:

  • Running multiple build tasks in parallel.
  • Replacing falsify Jenkins instances automatically.
  • Spinning up and terminating the slaves based on the requirement and need, which leads to cost reduction.

There are a few ways to implement Jenkins scaling; perhaps the easiest one is to implement is Jenkins scaling on Kubernetes.

This article will answer certain questions regarding the scaling process on Kuberenetes, including:

  • How is a slave pod created when a master generates a build?
  • How is the build assigned to the newly created slave?
  • How does the slave communication happens with the master during and after build?

Jenkins Master Installation

Prerequisites:

  • Jenkins Docker file
  • Kubernetes-Jenkins-deployment.yaml file
  • Kubernetes-Jenkins-service.yaml file
  • Kubernetes persistent volume
  • Kubernetes persistent volume claim ref:blazer.com

The preferred way to install Jenkins on Kubernetes is to create a custom Docker image based on the Jenkins base image. For now, you need to generate a Docker file that has the base image taken from Docker public repo, on which you can fine-tune your Jenkins application by adding plugins and installing Maven (RUN command) and alternative dependencies of your build along with Jenkins deployment.

The Docker file will prepare the image such that you don’t need to install any plugins after Jenkins deployment—it’s all done when the Jenkins Docker image is created.

Create a Jenkins-deployment.yaml file with deployment name, Docker custom Jenkins image, persistent volumes and other inputs, then deploy the application.

Since each deployment creates a pod, let’s check the status of the pod and how the Jenkins was deployed on the Kubernetes cluster.

Now, let’s create the service file for the Jenkins deployment to access the dashboard.

From the above image we can access the Jenkins dashboard with node ip and port number generated by the service file (32683).

Slave Configuration on Jenkins

The moment the Jenkins dashboard is up, checking to make sure the Kubernetes plugin is installed. If plugin installation is done, it will provide the options to configure slave details in  Manage Jenkins–>Configure System–>Kubernetes–>Add Cloud

Follow the below steps:

Then go to  Manage Jenkins–>Configure global security–>agents–>check the protocols JNLP agents fixed:50000 and (Java Web Start Agent Protocol/1) as mentioned in the below image.

Now, create a freestyle job and in the general tab check “Restrict where this project can be run” and give the slave-name that we have configured in the settings:

Workflow

  • When the job is triggered from master it will look for the slave configuration details.
  • Based on the configuration given in master using the JNLP Jenkins slave image, a slave pod is created and assigned to the build generated from the master until the build gets completed.
  • On a successful build, the slave will check for other build schedulers for a certain period of time. If there is no response from the master, the slave will post existing build results to the master and terminates.

— Lucky Kumar Sappa

Filed Under: Blogs, Continuous Delivery, DevOps Practice

« Next-Generation Technologies Drive Widespread Adoption of Modern IT Service Delivery Solutions
Successfully Integrating DevOps while Deploying Cloud Applications »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Cache Reserve: Eliminating the Creeping Costs of Egress Fees
Thursday, March 23, 2023 - 1:00 pm EDT
Noise Reduction And Auto-Remediation With AWS And PagerDuty AIOps
Thursday, March 23, 2023 - 3:00 pm EDT
Build Securely by Default With Harness And AWS
Tuesday, March 28, 2023 - 1:00 pm EDT

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Postman Releases Tool for Building Apps Using APIs
March 22, 2023 | Mike Vizard
What DevOps Leadership Should Look Like
March 22, 2023 | Sanjay Gidwani
Things We Should Acknowledge, Part One: Hiring Sucks
March 22, 2023 | Don Macvittie
HPE to Acquire OpsRamp to Gain AIOps Platform
March 21, 2023 | Mike Vizard
Oracle Makes Java 20 Platform Generally Available
March 21, 2023 | Mike Vizard

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

Large Organizations Are Embracing AIOps
March 16, 2023 | Mike Vizard
What NetOps Teams Should Know Before Starting Automation Journeys
March 16, 2023 | Yousuf Khan
DevOps Adoption in Salesforce Environments is Advancing
March 16, 2023 | Mike Vizard
Grafana Labs Acquires Pyroscope to Add Code Profiling Capability
March 17, 2023 | Mike Vizard
How Open Source Can Benefit AI Development
March 16, 2023 | Bill Doerrfeld
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.