Jenkins Pipeline are great. They allow the developer to keep their pipeline and their build near their code.

But developing a Pipeline can quickly become cumbersome:

  • The text area of the Jenkins UI is not really convenient when you start writing pipelines a bit more complex than your basic hello world.

  • Your code can grow and become hard to maintain (yeah, we end up having the same problems in our builds than in the rest of our code!).

  • Validation is not easy…​

This post will sum up some technics I found useful to help me be efficient with pipelines. Of course the list is not exhaustive. I am more than interested by any new technic you would like to share in the comments.

Writing the Jenkinsfile

This part will be focused on the tools you can use to assist you. As already mentionned, the Jenkins UI is not really the best place to comfortably write a huge script. Nevertheless, never forget its existence. When I have doubts, I often create a quick dummy job to validate/invalidate some part of a script.

The IDE

Well, I won’t go into an IDE war, this is far from being the point of the article. Your favorite java IDE (with potentially a groovy support plugin) will do the trick.

That being said, you can do more than having a simple groovy IDE, you can also teach your IDE to understand the DSL used by Jenkins! This works thanks to the GroovyDSL (GDSL). A GDSL is basically a file that will describe the meta language that a DSL uses. By putting this file in the classpath of a groovy project, your IDE will be able to assist you write code that follows the DSL. The good news is that the Jenkins pipeline exposes a GDSL. Go to $jenkins_base_url/pipeline_syntax/gdsl to get it. Then add this file in the classpath of your project and voilà, your IDE can now autocomplete!

Note that I only tested this feature in Intellij, but this should work in other groovy compatible IDEs. Also, plugins might enrich the DSL without being described in the GDSL, so don’t expect the autocomplete to be absolutely complete.

The linter

During development of a pipeline, it’s easy to make a typo in your Jenkinsfile and end up with an invalide Jenkinsfile. This is a pain, as you lose time understanding the mistake, fix it, commit the fix and push it. The good news is that there is a way to shorten this feedback loop.

Jenkins offers a linter for your Jenkinsfile. Best way to use the linter is with the JenkinsCLI. You will need the Pipeline Model Definition version 0.7 at least. You can then ask Jenkins to lint your Jenkinsfile:

curl -O $jenkins_base_url/jnlpJars/jenkins-cli.jar
java -jar jenkins-cli.jar -s $jenkins_base_url login --username $username --password $password
java -jar jenkins-cli.jar -s $jenkins_base_url declarative-linter<Jenkinsfile

The Pipeline Syntax

Even with the IDE assisting you, it is sometimes hard to correctly write what you wish to do:

  • Plugins are not exposed in the GDSL

  • The syntax might be cumbersome

  • Some parameters might seem obscure

There is a Pipeline Syntax page in Jenkins that is here to:

  • Help you understand the different options you have

  • Generate some code snippets for you

You can reach this page by browsing $jenkins_base_url/pipeline_syntax. I won’t go into details, the usage of the page is pretty self explanatory, you choose the step you wish to generate, then you fill in the options as you would in a good old freestyle job (click on the quotemarks if you need assistance), and finally you click the Generate Pipeline Script button to get your snippet. On the following screenshot, I generated a junit step. Note the presence of the help button (right red arrow) that allowed me to quickly understand the 'Retain long standard output/error' (which finally translated into keepLingStdio in the code):

Pipeline Syntax Page

The final part of the Pipeline Syntax page (circled in red in the screenshot) is a reference to the many variables you have access to in the pipeline.

Like for the GDSL, plugin sadly do not always support that page.

Best Practices

CloudBees proposed a good article with ten best practices for Jenkins Pipeline Plugin. This should help you getting started on the right track.

Organizing your pipeline

With pipelines, you can (and you should!) now treat your build as you treat your code! This means for instance that you should keep your pipeline DRY.

Clean coding can now be applied to your pipelines!

Extracting functions

As for the rest of the code, try to extract methods and reuse your code. Do not just copy and paste. Of course functions will not help you from a pipeline to another, but it is a good starting point. If you have the same function duplicated in multiple pipelines, it will help you to go to the next step. If the logic is duplicated throughout your pipelines, the next step will take more time (and will be more error prone).

Pipeline Shared Libraries

The Pipeline Shared Libraries Plugin allows a developer to share and reuse code between pipelines. I won’t go into details on how to use it here, it is worth a complete article, but here is a simple example that I encountered that will convince you of its power. All our builds had to notify an SCM. The only way to correctly catch all the cases (build success, build error), was by building inside a try/catch block. So we ended up starting all our builds duplicating the following logic:

#!/usr/bin/groovy

try {
    notify(start)
    //put here all your pipeline
    notify(success)
} catch(exception) {
    notify(failure)
}

def notify(state) {
    //notify scm
}

As you can see, there is a lot of duplicated code between the builds, with a huge maintenance problem.

What is happening is the SCM API is changing for instance? I would need to iterate over all the pipelines to update the code of the notify function.

Same issue if we can finally get rid of this try/catch structure (side note, that might soon be the case as we will see in the last part of this article). I would have to manually change all the Jenkinsfiles.

With the Shared Libraries Plugin, I was able to enrich the DSL with our particular use case. In the library I wrote:

//vars/notifiableBuild.groovy
#!/usr/bin/groovy

def call(body) {
    try {
        notify(start)
        body()
    	notify(success)
    } catch(exception) {
        notify(failure)
    }
}

def notify(state) {
    //notify scm
}

And finally, in a Jenkinsfile, I end up writing:

#!/usr/bin/groovy

notifiableBuild {
    //put here all your pipeline
}

To learn more about this, you should visit the official documentation of the plugin, which will help you getting started.

The Experimental Corner

In this part, I will share two projects you should keep an eye on as they should help a bit more spreading the usage of the pipelines. Note that the two projects are at a very early stage of development.

Jenkins Blue Ocean Pipeline Editor Plugin

This plugin is using the Blue Ocean interface. Side note, if you don’t know Blue Ocean, you should definitively have a look as it is far superior to the Jenkins 'classic' interface to display pipelines. Note that it is also work in progress (it is in a beta stage), but I use it in production everyday without any issue (except for the missing features of course).

That being said, this new Pipeline Editor plugin proposes a new interface to graphically design your pipeline. This is at very alpha stage, but it was demonstrated in the last Jenkins Meetup (note that the rest of the meetup is also worth your time if you are into Jenkins) and seems very promising.

Declarative Pipeline

The official name of this plugin is the Pipeline Model Definition Plugin. Its aim is to give a more config like touch to the pipeline. Code vs Config, here we are again (let’s not replay maven vs gradle and gulp vs grunt all over again…​)! Again, the plugin was demoed in the last Jenkins Online Meetup. The API does not seem stable for the moment, but some stuff seems very interesting. For instance, the notification issue I raised before could be adressed with something like:

#!/usr/bin/groovy

post {
    always {
    }

    success {
    	notify(success)
    }

    failure {
    	notify(failure)
    }
}

That’s it for this post, do not hesitate to post any comment you might have!

Spread the word

comments powered by Disqus