Jenkins Plugin for Distributed Workflows at Scale

Jenkins Plugin to accelerate and scale Continuous Delivery

Need to scale your Jenkins Distributed Workflows? Release Engineer’s Jenkins Plugin can do the heavy lifting.

I’ve had several people ask me how Release Engineer scales Jenkins distributed workflows to support a deployment that requires thousands of endpoints. This questions is mostly important to environments who do not want to rely on Jenkin’s slaves, have thousands of endpoints in a software deployments and have had difficulty implementing continuous deploy.  In addition, this questions comes up around distributed build workflows were Jenkins is sending out workflow steps across slaves.

Both Release Engineer and Meister can help scale a Jenkins distributed workflow using their Jenkins plugin.  First, Release Engineer can be called by Jenkins, and it is agentless.  When you perform a software deployment with Release Engineer, there is no need to install agents on every endpoint, virtual machine or container.  You use Jenkins to execute your continuous delivery process, but instead of executing a one-off deploy script, you hand over the heavy lifting to Release Engineer. Release Engineer manages the low level logic of the deployment and infrastructure configuration management and performs the delivery reporting back to Jenkins all critical information and reports.

In the same way, Meister can be called by Jenkins to execute a more complex compile/link/test workflow that must be done across multiple servers.  Meister’s workflow engine performs these lower level tasks and incorporates all of the load balancing and server pooling needed to maximize the efficiency of large distributed continuous integration processes.


Server pools for Jenkins distributed workflows

Remote Agent Server Pools

Because we wanted to get down to the real numbers, we did some heavy stress testing around large continuous deploy tasks. Check out the whitepaper for the results. This also addresses how to connect Release Engineer to Jenkins in order to off load the heavy lifting from the Jenkins CI Server. And yes, this is done WITHOUT AGENTS! No proper software deployment (ARA) solution should use agents at this point – don’t you agree?

Leave a Reply

Your email address will not be published. Required fields are marked *