Start "Rocky Linux" container with systemd in the Jenkins K3s cluster

111 Views Asked by At

I have Jenkins pipeline script like this which defines agent yaml and should run RockyLinux 8.6 with systemd service + installs some tools (tried different commands and installed missing tools, initialy tested with 8.9 Rocky Linux - but looks like from 8.7 version systemd service was removed (link: https://github.com/systemd/systemd/issues/27340) from image so started to use v8.6):

pipeline {
  parameters {
    choice(name: 'ROCKY_LINUX_IMAGE', choices: ['rockylinux:8.6'], description: 'Rocky Linux image to use.')
  }
  options {
    buildDiscarder(logRotator(numToKeepStr: '10', artifactNumToKeepStr: '10'))
  }
  // Wiki about docker container with systemd https://github.com/eniocarboni/docker-        rockylinux-systemd?tab=readme-ov-file
  agent {
    kubernetes {
      yaml(
        'apiVersion: v1\n' +
        'kind: Pod\n' +
        'spec:\n' +
        '  containers:\n' +
        '  - name: rockylinux\n' +
        '    image: ' + params.ROCKY_LINUX_IMAGE + '\n' +
        '    command:\n' +
        '    - "/usr/lib/systemd/systemd"\n' +
//        '    - "/usr/bin/dbus-daemon --system"\n' +
//        '    - "/usr/sbin/init"\n' +
        '    imagePullPolicy: IfNotPresent\n' +
        '    resources:\n' +
        '      limits:\n' +
        '        memory: "64Gi"\n' +
        '        cpu: "8"\n' +
        '    securityContext:\n' +
        '      privileged: true\n' +
//        '      capabilities:\n' +
//        '        add:\n' +
//        '        - CAP_SYS_ADMIN\n' +
        '  volumes:\n' +
        '  - name: systemd-units\n' +
        '    hostPath:\n' +
        '      path: /etc/systemd/system\n' +
        '  volumeMounts:\n' +
        '  - name: systemd-config\n' +
        '    mountPath: /etc/systemd/system\n'
      )
    }
  }
  stages {
    stage('Starting Container') {
      steps {
        script {
          echo "\n=================================\nNode Name: ${NODE_NAME}\nBuild Number: ${env.BUILD_NUMBER}\"
        }
        container('rockylinux') {
          sh "echo Hello World from ${params.ROCKY_LINUX_IMAGE}"
          sh 'whoami'
          script {
            echo 'Installing missing OS packages for CONTAINER'
            //sh 'modprobe fq_codel'
            sh '''
              export DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket
              yum -y install sudo passwd systemd firewalld dbus
            '''
            //sh 'systemctl start dbus'
            echo 'Installing openssh'
            sh 'sudo yum -y install openssh-server openssh-clients'
          }
        }

      }
    }
  }
  
}

}

Now I getting error like this:

Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: adeaafd3-fcc3-468b-886e-15dbd1646bf7 groovy.lang.MissingPropertyException: No such property: sh for class: groovy.lang.Binding at groovy.lang.Binding.getVariable(Binding.java:63) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:285) at org.kohsuke.groovy.sandbox.impl.Checker$7.call(Checker.java:375) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:379) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:355) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:355) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:355) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:355) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:29) at org.jenkinsci.plugins.workflow.cps.LoggingInvoker.getProperty(LoggingInvoker.java:121) at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20) at WorkflowScript.run(WorkflowScript:241) at cps.transform(Native Method) at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.get(PropertyishBlock.java:73) at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30) at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.fixName(PropertyishBlock.java:65) at jdk.internal.reflect.GeneratedMethodAccessor504.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:97) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Finished: FAILURE

Not sure what is the issue and how I could fix this. something with sh, also I imagine that it is related to YAMl executed "commands" as if I comment out last non failed steps - container still fails.

kubectl version Client Version: v1.29.0+k3s1 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.29.0+k3s1

Appreciate any help, thanks.

1

There are 1 best solutions below

0
On

So managed to make it work :).

Main issue was I think that YAML was not well formatted, also had to remove some properties, commands from jenkins pipeline.

My working YAML now looks like this:

pipeline {
  parameters {
    choice(name: 'ROCKY_LINUX_IMAGE', choices: ['rockylinux:8.6'], description: 'Rocky Linux image to use.')
  }
  options {
    buildDiscarder(logRotator(numToKeepStr: '10', artifactNumToKeepStr: '10'))
  }
  // Wiki about docker container with systemd https://github.com/eniocarboni/docker-        rockylinux-systemd?tab=readme-ov-file
  agent {
    kubernetes {
      yaml """
        apiVersion: v1
        kind: Pod
        spec:
          containers:
          - name: rockylinux
            image: ${params.ROCKY_LINUX_IMAGE}
            command:
            - /usr/lib/systemd/systemd
            imagePullPolicy: IfNotPresent
            resources:
              limits:
                memory: 64Gi
                cpu: 8
            securityContext:
              privileged: true
              capabilities:
                add:
                - CAP_SYS_ADMIN
          volumes:
          - name: systemd-units
            hostPath:
              path: /etc/systemd/system
          volumeMounts:
          - name: systemd-config
            mountPath: /etc/systemd/system
      """
    }
  }
  stages {
    stage('Starting Container') {
      steps {
        script {
          echo "\n=================================\nNode Name: ${NODE_NAME}\nBuild Number: ${env.BUILD_NUMBER}\"
        }
        container('rockylinux') {
          sh "echo Hello World from ${params.ROCKY_LINUX_IMAGE}"
          sh 'whoami'
          script {
            echo 'Installing missing OS packages for CONTAINER'
            sh 'yum -y install sudo passwd systemd firewalld dbus procps-ng'
            echo 'Installing openssh'
            sh 'sudo yum -y install openssh-server openssh-clients'
          }
        }

      }
    }
  }
}