Configure Seed Jobs and Pipelines

To preserve your Jenkins® pipelines and automate their recreation in case of instance failures, we strongly recommend using configuration as code files to set up pipelines.

Operator Service for Jenkins uses job-dsl and kubernetes-credentials-provider plugins for configuring jobs and deploy keys.

We have to start by preparing pipeline and job definitions in your project’s GitHub repository main folder using the following structure:

cicd/
├── jobs
│   └── build.jenkins
├── pipelines
│   └── build.jenkins

Jenkins will always check the configurations directly from those files in your repository, so you don’t have to update the configuration every time you change the pipeline code itself.

A seed job represents Jenkins job that creates one or more pipelines in Jenkins. It uses pipeline configuration files from your GitHub cicd folder. Let’s create a job configuration file and store it at https://github.com/your-project-repo/cicd/jobs/build.jenkins:

#!/usr/bin/env groovy

pipelineJob('build-operator-service-for-jenkins') {
    displayName('Build Operator Service for Jenkins')

    definition {
        cpsScm {
            scm {
                git {
                    remote {
                        url('https://github.com/your-account/your-repo.git')
                    }
                    branches('*/master')
                }
            }
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

• pipelineJob – name of pipeline resource that we will create in a while
• displayName – name of seed job that will be displayed in Jenkins UI
• remote – here you specify the url of GitHub repository of your project and username you are using in Operator Service for Jenkins, in order to access this repository
• branches – branches from which you want to access the repo
• scriptPath – the path in the above repo in which you store pipeline files from which you want to create Jenkins pipelines during this job run

Now we can create the pipeline configuration file and store it at https://github.com/your-project-repo/cicd/pipelines/build.jenkins. This file will create pod with containers to run commands on each step (stage) in your pipeline.

#!/usr/bin/env groovy

def label = "jenkins-example-${UUID.randomUUID().toString()}"

podTemplate(label: label,
        containers: [
                containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent:alpine'),
        ],
        ) {
    node(label) {
        stage('Init') {
            timeout(time: 3, unit: 'MINUTES') {
                checkout scm
            }
        }
        stage('Dep') {
            echo 'Hello from Dep stage'
        }
        stage('Test') {
            echo 'Hello from Test stage'
        }
        stage('Build') {
            echo 'Hello from Build stage'
        }
    }
}

• label – here specify the name of the pipeline (pipelineJob value from Seed Job)
• podTemplate – a pod that will be created during this pipeline run, to execute necessary steps
• containers – containers in a pod that will be used to run necessary steps. One jnlp container is always needed. All the others need to use images of programmes needed for the pipeline’s stages. Full list of possible functionalities can be found here. If you need to run kubectl commands you need to use container with an image that incorporates kubectl, because it is not provided by default.
• stage – at each stage you specify stage name, scripts, commands and container which needs to run them

When you create a seed job and pipeline files, you need to reference it and specify its details in the JenkinsSeedJob Custom Resource. Operator Service will create a default JenkinsKubernetesAgent “operator-agent” and seed job will be processed by default agent. Jenkins will then create Jenkins jobs from seed job files from your git repository and if you run it in the Jenkins UI, they will create the necessary pipelines.

apiVersion: operator-service.com/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    operator-service.com/jenkins: example
spec:
  repository:
    url: https://github.com/jenkinsci/kubernetes-operator.git
    branch: master
    targets: "cicd/jobs/*.jenkins"

Note: you have to specify the Jenkins Custom Resource name in the label operator-service.com/jenkins to connect the Seed Job with relevant Jenkins. It’s especially important in case of running multiple Jenkinses.

If you want to use your own Kubernetes Agent for seed job, you need to add AgentRef to JenkinsSeedJob:

apiVersion: operator-service.com/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    operator-service.com/jenkins: example
spec:
  repository:
    url: https://github.com/jenkinsci/kubernetes-operator.git
    branch: master
    targets: "cicd/jobs/*.jenkins"
  agentRef:
    name: agent-name
    namespace: default

Operator Service for Jenkins will then automatically discover and configure all the seed jobs. You can verify if deploy keys were successfully configured in the Jenkins Credentials tab.

If your GitHub repository is private or you need to authenticate to any other applications, you have to configure SSH or username/password authentication.

Using SSH Keys is a more secure option, while username/password method is good enough with solutions incorporating central location redirecting users for authentication or with multistep authentication.

To generate a private/public pair of keys we need to type the following command into our repo

$ ssh-keygen -t rsa -b 2048 -C "johndoe@email.com"
Generating public/private rsa key pair.

Next, you will be asked for a file name. Use the default path:

Enter file in which to save the key (/Users/johndoe/.ssh/id_rsa):

Now you can optionally set a password. In that case leave the password empty.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/johndoe/.ssh/id_rsa.
Your public key has been saved in /Users/johndoe/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:M0HppoJgPAhw2NYS0SVuYpyllpA7MbFfu+U0F0y8EDA johndoe@email.com
The key's randomart image is:
+---[RSA 2048]----+
|      o. .   o   |
|     o .o + +    |
|   ...+o . + .   |
|  E .ooo .. .    |
|   o  + S . .o   |
|  .  o   . o. +  |
| .    .   .o.=   |
|  . .... .o.@..  |
| .oo.o .o+=@*=   |
+----[SHA256]-----+

We need PEM format:

$ ssh-keygen -p -f /Users/johndoe/.ssh/id_rsa -m pem

Key has comment 'johndoe@email.com'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

Now we have to copy the content of the public key file

$ pbcopy < /Users/johndoe/.ssh/id_rsa.pub

and add it to our GitHub repository.

In your project repository enter Settings > Deploy keys and click “Add deploy key”. Give it some title and paste our copied key.

We can see the key was added.

First, create a Secret file with your GitHub username and generated SSH private key.

Copy contents of the id_rsa file (not id_rsa.pub):

$ pbcopy < /Users/johndoe/.ssh/id_rsa

$ xsel -b -i < /Users/johndoe/.ssh/id_rsa

And paste it into privateKey field like this:

apiVersion: v1
kind: Secret
metadata:
  name: k8s-ssh
  labels:
    "operator-service.com/jenkinsseedjob": "example-jenkins-seedjob"
    "operator-service.com/credentials-type": "basicSSHUserPrivateKey"
stringData:
  privateKey: |
   -----BEGIN PRIVATE KEY-----
    MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO
    oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8
    ...   
  username: github_user_name

Note: you have to specify the name of the JenkinsSeedJob Custom Resource in the labels to connect the secret with respective Seed Job.

Second, create a Kubernetes Secret resource from this Secret config file.

$ kubectl -n jenkins apply -f mySecret.yaml

In the seedJob you added to your JenkinsSeedJob Custom Resource file you can specify basicSSHUserPrivateKey as credentialType and add the name of the Secret, with your GitHub username and SSH key, in credentialID field’s value.

apiVersion: operator-service.com/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    operator-service.com/jenkins: example
spec:
  repository:
    url: git@github.com:your-account/your-repository.git
    branch: master
    targets: "cicd/jobs/*.jenkins"
    credentialType: basicSSHUserPrivateKey
    credentialID: k8s-ssh

Now we need to specify newly created credentials in your Seed Job definition file. They will be used to connect to specified github repository. Don’t forget to also change the url for SSH:

#!/usr/bin/env groovy

pipelineJob('build-operator-service-for-jenkins') {
    displayName('Build Operator Service for Jenkins')

    definition {
        cpsScm {
            scm {
                git {
                    remote {
                        url('git@github.com:your-account/your-repo.git')
                        credentials('k8s-sh')
                    }
                    branches('*/master')
                }
            }
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

First, create a Secret file with your GitHub credentials.

apiVersion: v1
kind: Secret
metadata:
  name: k8s-user-pass
  labels:
    "operator-service.com/jenkinsseedjob": "example-jenkins-seedjob"
stringData:
  username: github_user_name
  password: password_or_token

Second, create a Kubernetes Secret resource from this file.

$ kubectl -n jenkins apply -f mySecret.yaml

In the seedJob you added to your Jenkins Custom Resource file you can specify usernamePassword as credentialType and add the name of the Secret, with your GitHub credentials, in the credentialID field’s value.

apiVersion: operator-service.com/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    operator-service.com/jenkins: example
spec:
  repository:
    url: https://github.com/your-github-account/your-github-repository.git
    branch: master
    targets: "cicd/jobs/*.jenkins"
    credentialType: usernamePassword
    credentialID: k8s-user-pass