Jenkins mini project — migrating from one Jenkins server to another

Option 1 — using JCasC and Job Import

I will write out the steps here on how to migrate all configurations and builds/ jobs from an original Jenkins server to a second server using the Configuration as Code plugin and the Job Import plugin for Jenkins. 😄

My setup:

1. Two Jenkins servers installed on AWS EC2 Unbuntu 22.04 LTS instances.
2. The original Jenkins server is configured to integrate with LDAP.
3. The original Jenkins server is configured with Source Code Managementg to use GitLab.
4. The original Jenkins server also has some builds/ jobs created.
5. The second Jenkins server is just a fresh installation. (Note: any specific dependencies required by your builds/ jobs such as python3 packages etc. will need to be installed before hand, manually. If the EC2 instance creation is not automated. 😭)
   (Also note: some denpendecies installations may require restarting the jenkin service, which in turn will reload the configurations from /var/lib/jenkins/jenkins.yaml. So,make sure you put the latest configurations in the jenkins.yaml file.)

Step 1 Download and install the Configuration as Code plugin

On both the original Jenkins server and the second server, install the Configuration as Code plugin.

You can download the plugin from https://plugins.jenkins.io/configuration-as-code/releases/ — scroll down to the Installation options adn you will find the direct download link and the checksum link.

Check the checksum of the downloaded file:

• For Windows:
  certutil -hashfile [file location] SHA256
• For Linux:
  sha256sum [file location]
• For Mac OS:
  shasum -a 256 [file location]

To install the plugin, go to Dashboard -> Manage Jenkins -> Plugins -> Advanced settings and then scroll down to find the “Deploy plugin” section. Then you can specify the path to your downloaded file and click on the Deploy button.

After the installation, you can access and use the Configuration as Code plugin from Dashboard -> Manage Jenkins -> Configuration as Code.

Step 2 Export the configurations of the original server as code

Export the configurations of the original Jenkins server as code using the Configuration as Code plugin. 😄

Dashboard -> Manage Jenkins. You can either click on the Download Configuration button or simply click on the View Configuration button.

Before moving on to Step 3, there is Step 2.1 where you need to make a change to the code you exported. Specifically the below lines:

gitHubPluginConfig:
    hookUrl: "https://example.com/github-webhook/"

location:
    adminAddress: "address not configured yet <nobody@nowhere>"
    url: "https://example.com/"

Simply replace “example.com” with the URL address of your new Jenkins server.

Step 3 Import the configurations to the second server

What I did was simply copying the code from the original server (after the modification in Step 2.1) and paste it into a file called jenkins.yaml and storing it at the below location on the second server:

/var/lib/jenkins/

Then, I click on “Reload existing configuration” which should load the configurations from the file I just put under /var/lib/jenkins.

Note: From my experience, I found that the Configuration as Code plugin only “migrates” your credentials (such as private keys, secret files and secure text) under Dashboard -> Manage Jenkins -> Credentials. Also, in my situation, the LDAP configurations. To migrate all your jobs, I use the “Job Import” plugin.

It might also worth mentioning that JCasC is more useful when exporting code and use it as a reference rather than importing it directly into another Jenkins instance.

Step 3.1

Note if your builds/ jobs integrate with and pull from GitLab, you may need to manully generate an key pair and then add the public key to the specific GitLab project’s Deploy keys section, and add the private key to your Jenkins server GUI’s Credentials section (Dashboard -> Manage Jenkins -> Credentials.) OR just use the existing key pair from your original Jenkins server — manually add the private key to your Jenkins server GUI’s credentials section (Dashboard -> Manage Jenkins -> Credentials.

Step 4 Install the Job Import Plugin

Simply go to Dashboard -> Manage Jenkins -> Plugins -> Available plugins adn search for “Job Import”. You can then install it.

From here, you may need to edit your firewall rules, in my case, the AWS security group rules for my original server to allow the second server to connect to it on a chosen port.

Step 5 Configure Job Import

Go to Dashboard -> Manage Jenkins -> System and scroll down to “Job Import Plugin” section.

Give your original Jenkins server a name.

Put in the URL and port number in the Jenkins Server Url box. For example: http://example.com:8080/

For the Credentials box, click on Add and then provide the username and password and click on the Add button.

Then, click on the Credentials dropdown to choose the entry your just created.

Click on Save.

Go to Dashboard and you will see the Job Import Plugin option listed on the top left of the page.

On the Job Import Plugin page. Simple click on the “Query!” button. You should see all the builds/ jobs from the original server listed. You can now import them. Note: you may need to add a security group rule to allow port 443 access from the second Jenkins server to the original one.

Option 2 —Copy the jobs directories from the source to the target Jenkins server and Reload configuration from disk

# On the original server, copy the jobs’ directories toa separate folder I just created, one by one:

cp -r /var/lib/jenkins/jobs/[folder_with_job_name] .

# Secure copy them to the local machine

scp -r [original_server_ip]:~/[jobs_folder]/[job_name] .

# Then scp them from the local machien to the second server

scp -r [folder_with_job_name] [second_server_ip]:

# Change ownership of the copied folders recursively to jenkins:jenkins with the “-R” option

chown -R jenkins:jenkins add_staff_to_ldap

Note:Note: any specific dependencies required by your builds/ jobs such as python3 packages etc. will need to be installed manually.

PS: came across this blog post about Installing Jenkinson on an EC2 Instance which might be of interest. 😊https://medium.com/@ganasai88/installing-jenkins-on-an-aws-ec2-instance-66fbf72930a3