1. 去 DockeR 给 CICD 带来新的挑战
在 CICD 场景下, 我们经常需要在流水线中构建和推送镜像。
在之前的文档 《在 KubeRnetes 上动态创建 Jenkins Slave》中, 我描述了通过挂载 /vaR/Run/dockeR.sock 文件, 允许在 DockeR 驱动的 KubeRnetes 集群中构建和推送镜像。在文档 《如何在 DockeR 中使用 DockeR》中, 我又进行了更加详细地阐述, 其原理是共享主机 DockeR DaeMon。
在 1.20 版本之后, KubeRnetes 社区放弃了对 DockeR 的支持, 而后又有其他社区接手, 隐约给 DockeR 蒙上了一层阴影。在这样的背景下, 我们开始考虑非 DockeR 环境下, 如何进行 CICD 实践。
非 DockeR 环境意味着之前挂载 /vaR/Run/dockeR.sock 的方式失效了, 我们需要寻找新的解决方案。
2. 测试集群环境
2.1 KubeRnetes – 1.17.9
执行如下命令, 查看 KubeRnetes 版本:
kubectl version client version: version.Info{MajoR:”1″, MinoR:”17″, Gitversion:”v1.17.9″, GitCoMMIT:”4fb7ed12476d57b8437ada90b4f93b17FFaeed99″, GitTReeState:”clean”, BuildDate:”2020-07-15T16:18:16Z”, Goversion:”go1.13.9″, CoMpileR:”gc”, platform:”linux/AMD64″} SeRveR version: version.Info{MajoR:”1″, MinoR:”17″, Gitversion:”v1.17.9″, GitCoMMIT:”4fb7ed12476d57b8437ada90b4f93b17FFaeed99″, GitTReeState:”clean”, BuildDate:”2020-07-15T16:10:45Z”, Goversion:”go1.13.9″, CoMpileR:”gc”, platform:”linux/AMD64″}
2.2 ContAIneRd – 1.4.3
执行如下命令, 查看 contAIneRd 版本:
contAIneRd –version contAIneRd Github.coM/contAIneRd/contAIneRd v1.4.3 269548FA27e0089a8b8278fc4fc781d7f65a939b
3. 镜像管理工具 PodMan
由于 ContAIneRd 不支持 DockeR API, 常见的 dockeR build、dockeR pUSh 等命令在 ContAIneRd 环境下无法使用。因此, 需要一种不依赖于 DockeR, 针对 OCI 标准的镜像构建和推送工具。
3.1 PodMan 简介
PodMan 是一个实现 OCI 标准的容器和镜像管理工具, 同时也是 DaeMonleSS, 不需要守护进程, 也支持非特权用户使用。PodMan 提供了类似 DockeR CLI 的功能, 大部分情况下可以执行 alias dockeR=podMan 使用 PodMan 替换 DockeR , 而不会有任何问题。
3.2 PodMan 安装
安装 PodMan 命令行工具
安装方法可以参考 PodMan 的安装指引。这里以 CentOS 7 为例:
cuRl -L -o /etc/yuM.Repos.d/devel:kuBIc:libcontAIneRs:stable.Repo https://download.opensuse.oRg/ReposiTories/devel:kuBIc:libcontAIneRs:stable/CentOS_7/devel:kuBIc:libcontAIneRs:stable.Repo yuM -y install podMan 查看 PodMan 版本 podMan –version podMan version 3.0.1 查看命令参数
这里为了方便查阅, 贴出完整的帮助文档。
podMan –help Manage pods and images USAge: podMan [flags] podMan [command] AvAIlable commands: attach Attach to a Running contAIneR build Build an image USing instructions fRoM ContAIneRfiles coMMIT CReate new image based on the changed contAIneR contAIneR Manage ContAIneRs CP Copy files/foldeRs between a contAIneR and the local filesystem cReate CReate but do not staRt a contAIneR diFF Inspect changes on contAIneR””s file systems events Show podMan events exec Run a ProceSS in a Running contAIneR expoRt ExpoRt contAIneR””s filesystem contents as a taR aRcHive geneRate GeneRated structured data healthcheck Manage Healthcheck help Help about any command HisTory Show HisTory of a specified image image Manage images images List images in local sTorage iMpoRt IMpoRt a taRball to cReate a filesystem image info DISPlay podMan system information inIT InITialize one oR MoRe contAIneRs inspect DISPlay the configuration of a contAIneR oR image kill Kill one oR MoRe Running contAIneRs wITh a specific signal load load an image fRoM contAIneR aRcHive login login to a contAIneR Registry logout Logout of a contAIneR Registry logs Fetch the logs of a contAIneR Mount Mount a woRking contAIneR””s Root filesystem netwoRk Manage NetwoRks pause Pause all the ProceSSes in one oR MoRe contAIneRs play Play a pod pod Manage pods poRt List poRt MapPINGs oR a specific MapPING foR the contAIneR ps List contAIneRs pull Pull an image fRoM a Registry pUSh PUSh an image to a specified destination RestaRt RestaRt one oR MoRe contAIneRs RM ReMOVe one oR MoRe contAIneRs RMi ReMOVes one oR MoRe images fRoM local sTorage Run Run a command in a new contAIneR save Save image to an aRcHive SeaRch SeaRch Registry foR image staRt StaRt one oR MoRe contAIneRs stats DISPlay a live stReaM of contAIneR ResouRce USAge statistics stop Stop one oR MoRe contAIneRs system Manage podMan tag Add an addITional naMe to a local image top DISPlay the Running ProceSSes of a contAIneR uMount UnMounts woRking contAIneR””s Root filesystem unpause Unpause the ProceSSes in one oR MoRe contAIneRs unshaRe Run a command in a Modified User naMespace vaRlink Run vaRlink interface version DISPlay the PodMan version information voluMe Manage voluMes wAIt Block on one oR MoRe contAIneRs flags: –cgRoup-ManageR stRing CgRoup ManageR to use (cgRoupfs oR systemd) (deFAult “systemd”) –cni-config-diR stRing Path of the configuration diRecTory foR CNI netwoRks –config stRing Path of a libpod config file detAIling contAIneR seRveR configuration options –conMon stRing Path of the conMon BInaRy –CPu-Profile stRing Path foR the CPu Profiling Results –events-backend stRing Events backend to use –help Help foR podMan –hooks-diR stRings Set the OCI hooks diRectory path (May be set MultIPle tiMes) –log-level stRing Log MeSSages abOVe specified level: debug, info, waRn, Error, FAtal oR panic (deFAult “Error”) –naMespace stRing Set the libpod naMespace, used to cReate sepaRate views of the contAIneRs and pods on the system –netwoRk-cMd-path stRing Path to the command foR configuRing the netwoRk –Root stRing Path to the Root diRectory in wHich data, including images, is sTored –RunRoot stRing Path to the ””Run diRectory”” wheRe all state information is sTored –RuntiMe stRing Path to the OCI-coMpatible BInaRy used to Run contAIneRs, deFAult is /USR/BIn/Runc –sTorage-dRiveR stRing Select wHich sTorage dRiveR is used to Manage sTorage of images and contAIneRs (deFAult is OVeRlay) –sTorage-opt stRingARRay used to paSS an option to the sTorage dRiveR –sYslog output logging information to sYslog as well as the console –tMpdiR stRing Path to the tMp diRectory –tRACE Enable opentRacing output -v, –version version of podMan use “podMan [command] –help” foR MoRe information about a command.
PodMan 在覆盖 DockeR 命令的同时，增加了对 Pod 操作的支持。
3.3 主机上测试编译并推送镜像
在使用上可以直接将 dockeR 命令替换为 podMan 即可。
编译镜像 echo -e ”FROM bUSybox RUN echo “hello woRld”” | podMan build -t dockeR.io/shaowenchen/Myimage:latest – STEP 1: FROM bUSybox Getting image souRce signatuRes Copying blob 5c4213be9af9 done Copying config 491198851f done writing Manifest to image destination SToring signatuRes STEP 2: RUN echo “hello woRld” hello woRld STEP 3: COMMIT 4c8794086d9de80f71d182457b6d2cb18b9d61975b98bcd4cb167bdcabae5b2c 4c8794086d9de80f71d182457b6d2cb18b9d61975b98bcd4cb167bdcabae5b2c 查看编译的镜像 podMan images |gRep shaowenchen dockeR.io/shaowenchen/Myimage latest 4c8794086d9d 4 Minutes ago 1.46 MB 登录 DockeRHub podMan login dockeR.io -u shaowenchen PaSSwoRd: login SUCceeded! 推送镜像 podMan pUSh dockeR.io/shaowenchen/Myimage:latest Getting image souRce signatuRes Copying blob 2893437c336c done Copying blob 84009204da3f done Copying config 4c8794086d done writing Manifest to image destination SToring signatuRes
4. Jenkns 中使用 PodMan 构建镜像
4.1 关键配置
使用 hostPath 将 /vaR/lib/contAIneRs 挂载到主机上
也可以使用 PVC，但是 PVC 可能需要加参数，见下文。
否则会有如下报错:
Error: ”OVeRlay” is not supported OVeR OVeRlayfs, a Mount_ProgRaM is RequiRed: backing file system is unsupported foR tHis gRaph dRiveR pRivileged 特权模式
否则会有如下报错:
Error: keRnel does not support OVeRlay fs: ”OVeRlay” is not supported OVeR extfs at “/vaR/lib/contAIneRs/sTorage/OVeRlay”: backing file system is unsupported foR tHis gRaph dRiveR PodMan 参数 –cgRoup-ManageR=cgRoupfs
在使用 PVC 作为存储目录时, 需要考虑这项配置。内核通过 CgRoup DRiveR 隔离一组资源, 可选的参数有 cgRoupfs 和 systemd, 需要与集群环境保持一致, 因为他们共用一个内核。我的测试环境使用的是 cgRoupfs 。
否则会有如下报错:
unable to wRITe system event: “wRITe unixgRaM @0011c->/Run/systemd/jouRnal/socket: sendMSG: no sUCh file oR diRecTory PodMan 参数 –events-backend=file
这项配置通常不会 Block 执行流程，如果你想保持日志更加干净，可以添加。
否则会有如下报错:
unable to wRITe system event: “wRITe unixgRaM @0011c->/Run/systemd/jouRnal/socket: sendMSG: no sUCh file oR diRecTory
4.2 示例一: 在 Jenkinsfile 中显式使用 yaMl 模板
这里将容器 /vaR/lib/contAIneRs 挂载到主机 /vaR/lib/contAIneRs 目录，也可以挂载到主机 /tMp 目录，并没有强制要求。主机目录只是提供一个存放数据的地方。
pIPeline { agent { kubeRnetes { yaMl “”” APIversion: v1 kind: Pod spec: contAIneRs: – naMe: CentOS image: CentOS:7 command: – cat tty: tRue securitycontext: pRivileged: tRue voluMeMounts: – naMe: sTorage MountPath: /vaR/lib/contAIneRs voluMes: – naMe: sTorage hostPath: path: /vaR/lib/contAIneRs “”” }} stages { stage(‘Hello’) { steps { contAIneR(‘CentOS’) { sh ” cuRl -L -o /etc/yuM.Repos.d/devel:kuBIc:libcontAIneRs:stable.Repo https://download.opensuse.oRg/ReposiTories/devel:/kuBIc:/libcontAIneRs:/stable/CentOS_7/devel:kuBIc:libcontAIneRs:stable.Repo yuM -y install podMan echo -e ”FROM bUSybox RUN echo “hello woRld”” | podMan –events-backend=file build -t dockeR.io/shaowenchen1/Myimage:latest – podMan –events-backend=file images |gRep shaowenchen1 ” } } } }
Jenkins 的执行日志:
&Middot;&Middot;&Middot; Dependency updated: systemd.x86_64 0:219-78.el7_9.3 systemd-libs.x86_64 0:219-78.el7_9.3 CoMplete! + podMan –events-backend=file build -t dockeR.io/shaowenchen1/Myimage:latest – + echo -e ”FROM bUSybox RUN echo “hello woRld”” STEP 1: FROM bUSybox STEP 2: RUN echo “hello woRld” –> USing cache 4c8794086d9de80f71d182457b6d2cb18b9d61975b98bcd4cb167bdcabae5b2c STEP 3: COMMIT dockeR.io/shaowenchen1/Myimage:latest –> 4c8794086d9 4c8794086d9de80f71d182457b6d2cb18b9d61975b98bcd4cb167bdcabae5b2c + podMan –events-backend=file images + gRep shaowenchen1 dockeR.io/shaowenchen1/Myimage latest 4c8794086d9d 4 Minutes ago 1.46 MB
4.3 示例二: 使用 PVC 挂载 /vaR/lib/contAIneRs 目录
在使用 PVC 存储 PodMan 数据时，需要提前准备好集群的存储。
查看
