Jenkins 作为一款开源 CI/CD 工具,长久以来一直是自动化构建、测试和部署领域的“常青树”。尽管后起之秀不断涌现,但其强大的插件生态和灵活性使其在众多场景下依然是首选方案。今天,我们将探讨如何在 Kubernetes 环境中更优雅、更生产化地部署和使用 Jenkins。
本文将带你了解 CI/CD 的核心概念,剖析 Jenkins 在云原生环境下的优劣,并提供一个开箱即用的、基于 Configuration as Code (JCasC) 的 Kubernetes 部署方案。最后,我们还会通过具体的流水线示例,展示如何高效地利用它进行项目构建与部署。
什么是 CI/CD?
CI/CD 是一套旨在通过自动化实现软件频繁、可靠交付的现代实践方法。它主要包含三个核心环节:
持续集成 (Continuous Integration)
现代应用开发中,通常有多位开发者并行工作。传统“发版日”合并代码的方式不仅繁琐,而且容易出错。持续集成鼓励开发者频繁地将代码变更合并到主分支或特性分支。每次合并后,系统会自动触发构建,并运行自动化测试(如单元测试、集成测试)来验证这些变更,确保它们不会破坏现有功能。如果测试失败,CI 流程能帮助开发者快速定位并修复问题。
持续交付 (Continuous Delivery)
在 CI 阶段完成了自动化构建和测试后,持续交付能够自动将验证通过的代码制品发布到制品库(如 Nexus、Harbor)。其目标是始终拥有一个可随时部署到生产环境的软件包。在流程的最后,运维团队可以快速、轻松地将应用部署上线。
持续部署 (Continuous Deployment)
作为持续交付的延伸,持续部署可以自动将应用发布到生产环境。这高度依赖于精心设计的自动化测试,可能需要前期较大的投入来构建可靠的测试体系。
为什么在 Kubernetes 中选择 Jenkins?
优点
- 开源免费:零许可费用,对个人、初创公司和大型企业都极具吸引力。
- 强大的插件生态:拥有超过 1800 个社区插件,几乎能与任何技术栈和工具集成,这是其核心优势。
- 高度可定制和灵活:通过编写
Jenkinsfile 实现 Pipeline as Code,可以利用 Groovy 脚本定义复杂的构建、测试和部署流程。
- 庞大的社区与资源:历史悠久,拥有活跃的社区。遇到问题时,可以轻松从官方文档、Stack Overflow 等平台找到解决方案。
- 跨平台:基于 Java 开发,可运行在 Windows、Linux、macOS 等主流操作系统上。
- 分布式构建:支持 Master-Agent 架构,可将构建任务分发到多个代理节点执行,提高扩展性和并行处理能力。
缺点
- 配置与维护开销大:需要自行负责服务器的搭建、维护、升级、安全及插件兼容性管理,运维成本较高。
- 用户体验相对落后:与 GitLab CI、GitHub Actions 等现代工具相比,其 Web 界面显得较为陈旧。
- 插件管理是双刃剑:插件间可能存在依赖冲突,质量参差不齐,且是安全漏洞的主要来源。
- 有状态服务:配置、构建历史、日志均存储在服务器上,使得备份、迁移和容器化比无状态服务更复杂。
- Pipeline 语法学习曲线:编写复杂的
Jenkinsfile 需要学习 Groovy 语法和 Jenkins 特定的 DSL。
- 资源消耗较高:作为 Java 应用,尤其在管理大量任务和代理节点时,对内存和 CPU 有一定要求。
尽管有这些缺点,但通过在 Kubernetes 中采用声明式配置和容器化部署,可以显著缓解维护压力,并充分利用其灵活性。
在 Kubernetes 中部署 Jenkins:生产就绪方案
我们的目标是实现一个“部署即用”的 Jenkins 环境。方案核心包括:
- 使用 Configuration as Code (JCasC) 进行声明式配置。
- 增加 Sidecar 容器 监听配置变更,实现配置热重载。
- 预置多种语言的构建代理(Pod)模板,如 Java、Go、Node.js 等。
- 部署
dockerd DaemonSet,为流水线提供 Docker in Docker (DinD) 构建环境。
准备部署文件
以下是关键的 Kubernetes 资源配置清单。
ingress.yaml - 对外暴露 Jenkins 服务
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: jenkins
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
spec:
ingressClassName: nginx
tls:
- hosts:
- jenkins.kubeop.com
secretName: kubeop.com-ssl
rules:
- host: jenkins.kubeop.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: jenkins
port:
number: 8080
service.yaml - 定义 Jenkins Web 和 Agent 服务
---
apiVersion: v1
kind: Service
metadata:
name: jenkins
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
spec:
type: ClusterIP
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
---
apiVersion: v1
kind: Service
metadata:
name: jenkins-agent
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
spec:
type: ClusterIP
ports:
- name: agent-listener
port: 50000
protocol: TCP
targetPort: 50000
selector:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
rbac.yaml - 定义服务账户和角色权限
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jenkins-schedule-agents
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
- persistentvolumeclaims
- events
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- pods
- pods/exec
- persistentvolumeclaims
verbs:
- create
- delete
- deletecollection
- patch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jenkins-casc-reload
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
rules:
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- watch
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jenkins-schedule-agents
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins-schedule-agents
subjects:
- kind: ServiceAccount
name: jenkins
namespace: infra
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jenkins-watch-configmaps
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins-casc-reload
subjects:
- kind: ServiceAccount
name: jenkins
namespace: infra
secret.yaml - 存储管理员凭据
apiVersion: v1
kind: Secret
metadata:
name: jenkins
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
type: Opaque
data:
jenkins-admin-password: "YWRtaW4="
jenkins-admin-user: "YWRtaW4="
statefulset.yaml - Jenkins 控制器核心部署 (已精简,保留关键部分)
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: jenkins
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
spec:
serviceName: "jenkins"
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
template:
metadata:
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
spec:
serviceAccountName: "jenkins"
initContainers:
# Init Container 用于初始化目录和预下载插件
- name: init
image: registry.cn-hangzhou.aliyuncs.com/kubeop/jenkins:2.528.1
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
runAsGroup: 1000
runAsUser: 1000
command:
- /bin/bash
- -ec
- |
echo "disable Setup Wizard"
# Prevent Setup Wizard when JCasC is enabled
echo ${JENKINS_VERSION} > ${JENKINS_HOME}/jenkins.install.UpgradeWizard.state
echo ${JENKINS_VERSION} > ${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
echo "download plugins"
jenkins-plugin-cli --war /usr/share/jenkins/jenkins.war -f /usr/share/jenkins/ref/plugins.txt --plugin-download-directory ${JENKINS_HOME}/plugins --verbose;
volumeMounts:
- name: jenkins-data
mountPath: /data/jenkins
containers:
- name: jenkins
image: registry.cn-hangzhou.aliyuncs.com/kubeop/jenkins:2.528.1
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
runAsGroup: 1000
runAsUser: 1000
args: ["--httpPort=8080"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: JAVA_OPTS
value: "-Xms8192m -Xmx8192m -Dcasc.reload.token=$(POD_NAME) -Dfile.encoding=utf-8 -Duser.timezone=GMT+08 -Duser.country=CN"
- name: CASC_JENKINS_CONFIG
value: /data/jenkins/casc_configs
ports:
- containerPort: 8080
name: http
protocol: TCP
- containerPort: 50000
name: agent-listener
protocol: TCP
volumeMounts:
- name: jenkins-jcasc-config
mountPath: /data/jenkins/casc_configs
- name: admin-secret
mountPath: /run/secrets/admin-username
subPath: jenkins-admin-user
readOnly: true
- name: admin-secret
mountPath: /run/secrets/admin-password
subPath: jenkins-admin-password
readOnly: true
- name: jenkins-data
mountPath: /data/jenkins
# Sidecar 容器,监听 ConfigMap 变更并触发 Jenkins 配置重载
- name: config-reload
image: registry.cn-hangzhou.aliyuncs.com/devops-system/k8s-sidecar:1.30.7
imagePullPolicy: IfNotPresent
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: LABEL
value: "jenkins-config"
- name: FOLDER
value: "/data/jenkins/casc_configs"
- name: REQ_URL
value: "http://localhost:8080/reload-configuration-as-code/?casc-reload-token=$(POD_NAME)"
- name: REQ_METHOD
value: "POST"
volumeMounts:
- name: jenkins-jcasc-config
mountPath: /data/jenkins/casc_configs
volumes:
- name: jenkins-jcasc-config
emptyDir: {}
- name: admin-secret
secret:
secretName: jenkins
- name: jenkins-data
persistentVolumeClaim:
claimName: jenkins
local-pv.yaml - 本地持久化存储
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins
namespace: infra
spec:
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- worker-001
capacity:
storage: 100Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local
local:
path: /data/jenkins
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: jenkins
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
storageClassName: local
volumeMode: Filesystem
volumeName: jenkins
jcasc-config.yaml - Jenkins Configuration as Code 配置 (核心,已大幅精简结构)
这是通过 ConfigMap 定义的 JCasC 配置,它包含了安全设置、云配置(Kubernetes 插件)、预定义的 Pod 模板等。由于内容非常长,这里仅展示其结构和部分关键配置,以说明其强大能力。
apiVersion: v1
kind: ConfigMap
metadata:
name: jenkins-jcasc-config
namespace: infra
labels:
app.kubernetes.io/name: "jenkins"
app.kubernetes.io/component: "jenkins-controller"
jenkins-config: "true"
data:
jcasc-default-config.yaml: |-
credentials:
system:
domainCredentials:
- credentials:
# 定义各类凭据,如 SSH、用户名密码、Secret文件等
jenkins:
authorizationStrategy:
# 配置基于矩阵的权限策略
clouds:
- kubernetes:
name: "kubernetes"
serverUrl: "https://kubernetes.default.svc.cluster.local"
jenkinsUrl: "http://jenkins.infra.svc.cluster.local:8080"
jenkinsTunnel: "jenkins-agent.infra.svc.cluster.local:50000"
namespace: "infra"
containerCap: 60
templates:
- name: "java"
label: "java"
containers:
- name: "java"
image: "registry.cn-hangzhou.aliyuncs.com/kubeop/maven:3.9.11-java8"
command: "/bin/sh -c"
args: "cat"
ttyEnabled: true
resourceRequestCpu: "500m"
resourceLimitCpu: "4"
resourceRequestMemory: "512M"
resourceLimitMemory: "4096M"
- name: "jnlp"
image: registry.cn-hangzhou.aliyuncs.com/devops-system/inbound-agent:bookworm-jdk21
- name: "golang"
label: "golang"
containers:
- name: "golang"
image: registry.cn-hangzhou.aliyuncs.com/kubeop/golang:1.25.3
# ... 资源限制
- name: "jnlp"
# ... jnlp 容器配置
- name: "docker"
label: "docker"
volumes:
- hostPathVolume:
hostPath: "/var/run/docker.sock"
mountPath: "/var/run/docker.sock"
containers:
- name: "docker"
image: registry.cn-hangzhou.aliyuncs.com/kubeop/docker:28.5.1
privileged: "true"
# ... 资源限制
- name: "jnlp"
# ... jnlp 容器配置
# ... 更多模板如 nodejs, kubectl, ansible 等
securityRealm:
local:
users:
- id: "${admin-username}"
name: "Admin"
password: "${admin-password}"
# ... 其他全局配置如代理端口、视图、工具配置等
此配置通过 templates 定义了多种构建代理 Pod 模板。当流水线指定 agent { label 'java' } 时,Jenkins 的 Kubernetes 插件就会在指定的命名空间中动态创建一个包含 java 和 jnlp 容器的 Pod 来执行任务,任务结束后 Pod 自动销毁。这种模式完美契合了容器化和云原生的按需使用理念。
dockerd.yaml - 提供 Docker 构建环境
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: dockerd
namespace: infra
labels:
app.kubernetes.io/name: dockerd
spec:
selector:
matchLabels:
app.kubernetes.io/name: dockerd
template:
metadata:
labels:
app.kubernetes.io/name: dockerd
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: docker
operator: In
values:
- "true"
containers:
- name: dockerd
image: registry.cn-hangzhou.aliyuncs.com/kubeop/docker:28-dind
imagePullPolicy: Always
args:
- --insecure-registry=registry.kubeop.com
- --bip=192.168.100.1/24
- --data-root=/data/docker
securityContext:
privileged: true
volumeMounts:
- mountPath: /var/run
name: docker-sock
- mountPath: /data/docker
name: docker-data
volumes:
- name: docker-data
hostPath:
path: /data/docker
- name: docker-sock
hostPath:
path: /var/run
该 DaemonSet 在特定节点上运行 Docker Daemon,并通过挂载 hostPath 将 Docker Socket 暴露给 Jenkins 的 docker 代理 Pod,从而使流水线能够在容器内执行 docker build 命令。
部署步骤
- 创建命名空间和 TLS 证书
kubectl create ns infra
kubectl create secret tls kubeop.com-ssl --cert kubeop.com.pem --key kubeop.com.key -n infra
- 添加镜像拉取 Secret (如果需要从私有仓库拉取镜像)
kubectl create secret docker-registry hubsecret \
-n infra \
--docker-server='<DOCKER_SERVER>' \
--docker-username='<DOCKER_USER_NAME>' \
--docker-password='<DOCKER_USER_PASSWORD>' \
--docker-email='<DOCKER_USER_EMAIL>'
- 应用所有配置文件
kubectl apply -f . -n infra
部署完成后,访问配置的 Ingress 地址 (如 https://jenkins.kubeop.com) 即可使用预配置的管理员账户登录。
配置流水线
部署好 Jenkins 后,我们来创建两个不同复杂度的流水线示例。
简单流水线
适用于规模小、流程相对简单的项目。这是一个完整的 Jenkinsfile 示例,展示了代码拉取、SonarQube 扫描、Maven 构建、Docker 镜像打包和部署到 Kubernetes 的完整Pipeline。
#!/usr/bin/env groovy
node {
APP_PROJECT = JOB_NAME.split('_')[0]
APP_NAME = JOB_NAME.split('_')[1]
APP_WORKSPACE = JENKINS_HOME + '/workspace/' + JOB_NAME.toLowerCase()
DOCKER_REGISTRY = "registry.cn-shanghai.aliyuncs.com"
GIT_NAME = "https://github.com/kubeop/java-demo.git"
}
pipeline {
agent any
options {
timestamps()
timeout(time: 30, unit: 'MINUTES')
disableConcurrentBuilds()
buildDiscarder(logRotator(numToKeepStr: '10'))
}
parameters {
gitParameter(
branch: '',
branchFilter: 'origin/(.*)',
defaultValue: '',
listSize: '10',
name: 'SCM_REVISION',
quickFilterEnabled: true,
selectedValue: 'NONE',
sortMode: 'DESCENDING_SMART',
tagFilter: '*',
type: 'PT_BRANCH_TAG',
description: 'Please select a branch or tag to build')
choice(
name: 'ENVIRONMENT',
description: 'Please select Environment',
choices: 'dev\nfat\nuat\npro')
}
stages {
stage ("Initial Stages") {
steps {
script {
node ("built-in") {
dir(APP_WORKSPACE){
stage('stage 1: Git Clone') {
deleteDir()
checkout([$class: 'GitSCM',
branches: [[name: "$SCM_REVISION"]],
userRemoteConfigs: [[credentialsId: 'gitlab',url:"$GIT_NAME"]]])
if (!SCM_REVISION) { error "您没有选择Git分支或TAG!" }
wrap([$class: 'BuildUser']) { env.BUILD_USER = BUILD_USER }
currentBuild.displayName = BUILD_NUMBER + '-' + APP_ENV
currentBuild.description = BUILD_USER + ' deploy by ' + SCM_REVISION
}
}
}
node ("sonar") {
container("sonar"){
dir(APP_WORKSPACE){
stage('stage 2: Scanner Code') {
withSonarQubeEnv('sonar') {
sh 'sonar-scanner -Dsonar.projectKey=$APP_NAME -Dsonar.projectName=$APP_NAME -Dsonar.projectVersion=$GIT_REVISION -Dsonar.projectBaseDir=. -Dsonar.language=java -Dsonar.sources=. -Dsonar.java.binaries=.'
}
}
}
}
}
node ("java") {
container("java"){
dir(APP_WORKSPACE){
stage('stage 3: Compile Code') {
sh 'mvn -U clean -Dmaven.test.skip:true package dependency:tree'
}
}
}
}
node ("java") {
container("java"){
dir(APP_WORKSPACE){
stage('stage 4: Junit Test') {
sh 'mvn test'
//junit 'reports/**/*.xml'
}
}
}
}
node ("docker") {
container("docker"){
dir(APP_WORKSPACE){
stage('stage 5: Build image') {
docker.withRegistry("https://" + DOCKER_REGISTRY, 'harbor') {
docker.build(DOCKER_REGISTRY + "/" + APP_PROJECT + "/" + APP_NAME + ":" + GIT_REVISION,"-f Dockerfile . --pull")
docker.image(DOCKER_REGISTRY + "/" + APP_PROJECT + "/" + APP_NAME + ":" + GIT_REVISION).push()
}
}
}
}
}
node ("kubectl") {
container("kubectl"){
dir(APP_WORKSPACE){
stage('stage 6: Deploy to Kubernetes') {
sh 'kubectl set image deployment/' + APP_NAME + ' ' + APP_NAME + '=' + DOCKER_REGISTRY + '/' + APP_PROJECT + '/' + APP_NAME + ':' + GIT_REVISION + ' -n ' + APP_PROJECT + '-' + ENVIRONMENT
}
}
}
}
}
}
}
}
}
进阶流水线
对于规模较大、流程复杂的项目,推荐使用 Jenkins Shared Library 将通用步骤抽象成库,使 Jenkinsfile 极度简化,更专注于流程编排。
// 加载共享库
library(
identifier: 'jenkins-shared-library@main',
retriever: modernSCM(
[
$class: 'GitSCMSource',
credentialsId: 'devops',
remote: 'https://github.com/kubeop/jenkins-shared-library.git',
traits: [
gitBranchDiscovery()
]
]
)
)
// 调用共享库入口函数,传入参数即可
entry([
clean_workspace: true,
git_repo: "https://github.com/kubeop/java-demo.git",
build_command: "mvn",
build_options: "-U clean -Dmaven.test.skip:true package dependency:tree",
artifact_src: "target/demo-0.0.2.jar",
artifact_dest: "demo.jar",
docker_registry: "registry.cn-shanghai.aliyuncs.com",
docker_registry_credential: "harbor",
k8s_cluster: "ops-pro"
])
构建示例
创建流水线任务并构建时,可以看到清晰的阶段视图。以下是不同流水线执行时的阶段耗时示例。
构建参数界面示例:
用户可以在构建前选择代码分支、部署环境等参数。

流水线执行阶段视图示例:
这些图表展示了流水线各个阶段(如代码克隆、代码扫描、编译、构建镜像、部署到K8s等)的执行耗时,帮助进行性能分析和优化。



总结
通过在 Kubernetes 中采用本文所述的声明式部署方案,我们成功将 Jenkins 的运维复杂度降低。利用 JCasC 实现配置即代码,结合 Sidecar 容器实现动态重载;利用 Kubernetes Plugin 实现动态Pod代理,构建环境隔离且资源高效;最后通过 Shared Library 提升流水线的可维护性和复用性。
这套组合拳使得 Jenkins 这个“老将”在云原生环境中重新焕发活力,既能应对复杂的定制化需求,又具备了现代化工具的声明式管理和弹性伸缩能力。对于正在寻求稳定、可控且功能强大的 CI/CD 解决方案的团队,这无疑是一个值得深入实践的方向。如果你在云原生和 DevOps 自动化方面有更多的想法或问题,欢迎在云栈社区与大家交流探讨。