拆车五十铃FTR54发动(Gitlab+Jenkins Pipeline+k8s在生产环境中的应用)

Posted 2023-05-27

篇首语：认识到自己无知是知识进步的重要阶段。本文由小常识网(cha138.com)小编为大家整理，主要介绍了拆车五十铃FTR54发动(Gitlab+Jenkins Pipeline+k8s在生产环境中的应用)相关的知识，希望对你有一定的参考价值。

拆车五十铃FTR54发动(Gitlab+Jenkins Pipeline+k8s在生产环境中的应用)

系统环境说明

GitLab Community Edition 11.9.8
Jenkins ver. 2.190.3
仓库使用阿里的镜像仓库
Kubernetes v1.14.2

gitlab和jenkins-master可以选择自建或者部署到k8s中，当前场景是部署在k8s集群之外；
镜像仓库可以选择使用harbor或者阿里镜像仓库，当前场景使用的是阿里的镜像仓库；

编译发布流程

流程很简单，提交代码到不同的分支，触发通知到jenkins，jenkins pipeline会根据Jenkinsfile文件中定义k8s环境，动态生成一个jenkins slave在不同k8s环境中构建镜像，推送镜像到仓库，然后在部署到对应k8s环境，部署结束后jenkins slave会自动终止

集成配置过程

因为都是现有的环境，所以部署过程就省略了，直接开始做集成配置；如果你是全新安装的Jenkins，选择安装推荐插件+kubernetes插件；现在假如插件都已经安装完成了，jenkins登陆默认账户admin，密码查看/var/jenkins_home/secrets/initialAdminPassword文件，下面开始配置

点击左侧系统管理，打开系统配置，我们这里要实现部署发布到2个k8s环境，所以我们配置两个云，先新增一个云

名称：随便填，后面Jenkinfile会用到
Kuberneters地址：填写apiserver地址
Kuberneters服务证书key：需要拿k8s的crt和key做格式转换，下面会介绍
Kubernetes 命名空间：填写jenkins slave要生成的命名空间，这个自己看着填吧
凭据：需要拿k8s的crt和key做格式转换，然后生成jenkins全局凭据
Jenkins地址：填写jenkins master地址，也就是当前jenkins地址
Jenkins通道：填写jenkins master与jenkins slave通信地址,jenkins通信端口默认是50000

配置第一个k8s相关信息，过程中会用到认证k8s apiserver的key和凭证，所以我们先获取下需要的key和凭证（在要配置的k8s master机器操作）

$ cat /root/.kube/config apiVersion: v1clusters:- cluster: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1EWXdOakV3TWpZd09Wb1hEVEk1TURZd016RXdNall3T1Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSlBSCmFnUlVKVytleDlKeTFZOXEzUVpNZk0wWnFJbkxjOE43RFVnZnM4TktlUVh2SUFCNkxxdjBSNFY4VUNnYnZ6dEMKVitxdElGNUM5bmE5VFQzT3hVNkUwQnVmWTcwTmJBZ2dPN0RTN1FvQVc3ZG5HUnBDTmNieWg5dytZYi9vbkNCdgo3M28vRi9scnhFZ01jNFhuYTR6OGhXbm5STmdjcVBSVnNyWGFiVSt6TStsbVZEaEpwWE96dnVmMmZRb3creGF4CndaWnVwUmF5VDBESHVHbmpaQnkrNnFwQVdZampqaE9WOUhGcTlQQUpMUXAzR2xZdklueFgxUkJscTYyVFdZMW8KcjIvTFBRTUhjOUV6VFlVN21Qb0laQ3dqa1dPTzZmc1NFVHpBTk9ad1NlSlBRSW5XV1NlQXlsWjA4V2tNWjdVcQpDNkZHVml1REFVbE1HczBMMlhzQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFc1I5T2VSRW45NXdSaTA2UnI2SVhUcDhJeHkKSytmdFJyM1pZckZ5VWZZYTBWdU9Mc1NZdzByN1o5Zmk0ZUFlMEk4dnR2cWpqWWl6RzFnUFAyS3V4d0h4RmtJRApMQnlNRmRTSG5yVEVZeWo2NVFnbUtCWVpEZ0VkMnZpVnBTeHM4Y3dCZXgvT201VnErZnROanAwK2swaGdhV2xxCjBDZmtkbjM0MkY2bUhSZFNyeGg3eURleGNtNEtGck5OVnNPY0h5MEJhQXhwZ0JOUmErMG5oZ1dDbHh1M0F4OWgKaVRELzAyczRCcWZKTFZjZXk0Q2VnWW8zUDVDYWVjTTZSaTg4TVFKYlZ1OWx6RVZPUzlBRGNKZ0VkWkdHUFUyVwpEcHBhZXdZNjhSVVg1MG9ObXN2S2h1RGNCSWxyeHJ4T2J4Wk1wYm5hYldUMlFaKzM4ampEaDNwVmxhcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= server: https://192.168.0.54:6443 name: kubernetescontexts:- context: cluster: kubernetes user: kubernetes-admin name: kubernetes-admin@kubernetescurrent-context: kubernetes-admin@kuberneteskind: Configpreferences: users:- name: kubernetes-admin user: client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJSzZSZWRVQ2NNS0V3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RBMk1EWXhNREkyTURsYUZ3MHlNREEyTURVeE1ESTJNVEZhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTBrcW1jci9DNHBaSTgwUTAKTzBOaklRV1hPelQ0TmVPVFJQd0N1UW1CaDlZcUJ0QTBkZnJvVnRzT2JRTEdlWE5GUVlhRzM5V2ZsN292YURCNwp3Wm11WVo3alZKUm9BZmpldEg3d0lIN3pNYUdCbk1hN3RoTUNqWXBmdXRpQyszQm51cjFLRmFJR2FKcDNlK0NQCjAyVjdSVDJjUXFPbHVOVUtFSEl2UG15YTdKM0pHWUpLNFVBVEhQOVhIMVZNemVoc3N5ZjF5UTZZb1BxTTR1b08KVGVNN3g0NFcvS2hrWjBMWkhncnB5RDJQNlN2NGVoaGl6YTQrdUk1Si80OFhhVFJwMUxINmdGODVhalkwNXIybApQUW95dDFPTEdIVHZtRnA3ei9QMHBKbUJpcGQwcktBeFNJLzU5SU93elYyRFNpQmp4TjQwSWpKNWhidlczNHNFCmtHNTFTUUlEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFWGM0VUhiOVFMSisvU2t0S3dhOVpydVJ3THBhV0NVeE80Sgo1Mml0eHdZK3JLdE9WQVBsKzdNRjVmcWU3UUhmcW0yMjFmR1V6UW50VERpM1RjOVNsdUVDdjdxdHBCcHczZ1pYCnZpSmIvakRRTWwrdmhvZW95UlpiUmhWZG5kUU5lS1FoUzl4UEw3Rmx2d0pRUnNsWTlCdWJ4L3lsVC9KRElKcFcKRzJhanM4ZGNyWUlLeGlEbjdQSTZqeFB1OURRVUgraGVza1M5VXRwcUF3M3pBdjFCaG5kV2FEWUNQc293MnloVAp1RFNrWG9XVE9rTlQ2UDVWREdob3VZbzJ4emhJeDl4NWllMVpLOHlnODY5THZvZkJQNk5oRnZRNDRPQkFyR29JCnNJcDNJT0ZpaWJjVW1wVXNZNWJ2OEVXbjBNdXkxN1VjVTlvMkYvTnpuQ1hZV1JEVUViVT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMGtxbWNyL0M0cFpJODBRME8wTmpJUVdYT3pUNE5lT1RSUHdDdVFtQmg5WXFCdEEwCmRmcm9WdHNPYlFMR2VYTkZRWWFHMzlXZmw3b3ZhREI3d1ptdVlaN2pWSlJvQWZqZXRIN3dJSDd6TWFHQm5NYTcKdGhNQ2pZcGZ1dGlDKzNCbnVyMUtGYUlHYUpwM2UrQ1AwMlY3UlQyY1FxT2x1TlVLRUhJdlBteWE3SjNKR1lKSwo0VUFUSFA5WEgxVk16ZWhzc3lmMXlRNllvUHFNNHVvT1RlTTd4NDRXL0toa1owTFpIZ3JweUQyUDZTdjRlaGhpCnphNCt1STVKLzQ4WGFUUnAxTEg2Z0Y4NWFqWTA1cjJsUFFveXQxT0xHSFR2bUZwN3ovUDBwSm1CaXBkMHJLQXgKU0kvNTlJT3d6VjJEU2lCanhONDBJako1aGJ2VzM0c0VrRzUxU1FJREFRQUJBb0lCQUE3bXcweTJVZlVFZVQ3agp3bC9Bc3JHUVY5c1dNZEIvdzl2TGo5WFUycHpwakNqWGNDQThHMktzT3lWMllPSVNUUUlMcWxzS0pEajROSXZKCmc3dUFURjhXaHoxakZzdXMrdnNIVTdTNXlqbm1HKzBrR0FFYTc3OWY0dEMycnZGcVVhOWw0bTRPQVM1QVk5OGYKVnBIQVN5L280YjNISXVNcUZZQjgxdVF4aGZqbVJHTWJ3dFdRM090dUU1d2xQRDJwOFBqTElaQVNsVTZ2RlpoQQpMWCs0ZjIyMWRjSkxTVGhqTDFiVHFPM1NkazE3WjZOV1M1Ym9TMFptY0JSOFhLUHNqUHFVek5zRFVxeCs1cDdKCkFXVGNlYjB4dWZYTldDV3Y3N3NUcTBjdyt6UXNwbVZwbFkzMjA4d0c5Tm9FYm5hMGFNQ1ZacG1oK3BSR2xOelkKRlNZNU9rRUNnWUVBMGxLcjI2S3IvMU1KcGFGMHRtWWNHcVRzd093bklBMzBNemk2blN5SVd1Q21qRWs3ZU5zSgpTRUk5cWRPRzhhWGw3VGZpa2VYUko1b3MxcDFZZzh2TUdpam01NGI3ejYybFZRUWtsOWZPSlVGM1YzQkgyR2p2CjRpOFZVTUl5ZmNSRmZUUFZEeGtKbHZtdnZCS0kwZU9UV0daVmFxb1AzZ0d0SW5lVDlmbEgwUTBDZ1lFQS8vWTgKbzZaelRtejIyVWVXZW5aemlNOHBxRHBqVU5DRnlKemJqajNvN1ZsTGRSc0FybnpFZnVNaUo5YjZ2dVpaWXQzRwp2bEZFMDBiYnNQWmRVbzArenJvSEZMK2JqQ1QrODl1UUFJRHFHcGtyYUdIZ0IxelZyMHF2L0NXajlnbzkrdUxhCjJKQk5ud1RSczlsbzE1ZzV1YnJPRWFBNzlBaExEeEFuZFA3Z0RpMENnWUFyODBpa2NmN2RNUDRBRlpndEVYTm8KQWZUVGI4WFJSZmsweHZNQUt6RW5SSENwT2hocWJlTW5yV2Z6V0JlSDRiSUZlenNtWDg3d0pxQ2VERzFWeFQyVwpiZHVxb0NONHg0R1lIWENFSm4yV2ZYS3gyKzIzaEY0MGRzQk9pdlpBSDhhaG5qWTBuSGZMaTh1MFVtOHk1UXFDClc0Z1g3UWU1emNIZlJQdXZWL013OVFLQmdRQy8rKzFYd2cxU2thQkZNTkRKWWZjZWNtUUlibUwzeHEvUjNQVkIKSjJhQ1FDdTgxbGdZaURUS0I0c2kzcmlNWHpKRVdad3NPOENueDhvWVhYRjU3YjlpUjEzd1RoZFpjcFpZU2lNawpmWTBhRGpEa3hpVEc0UGJWMSt0UDhOdWVPK2hwT2FaME1TaEhVZElJVjlXdmY5b3NXTlVmbTFQY29pdktUSStMCnpYQTRzUUtCZ0ZsUnpoeDlYczJ2L1h1VGl0NE1mNUpMU2ppNk93c3pZVXNmTzVMQ1BVbGQrcjA5dmFsS0tPQlcKU1FUVDhTdnBpMzZhQVFTV0xCUUdVYkh2VlJ0QkJHOWxiY2JSSHlla1RrYksxTEwxVk0zKzdtRXZCUDFZZU9qZQpwWkRlTWcrZWQyY3F5MDJhZWwxbGhBK3dJRmlnUzV5bGY0Z25paFBnMVZ2WmJIUGIzREdGCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

我们用到的Kubernetes 服务证书转换 keycertificate-authority-data，凭据转换client-certificate-data和client-key-data

获取/root/.kube/config中certificate-authority-data的内容并转化成base64 encoded文件,将生成的ca.crt文件内容填写到jenkins kubernetes的Kubernetes 服务证书key中

$ echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1EWXdOakV3TWpZd09Wb1hEVEk1TURZd016RXdNall3T1Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSlBSCmFnUlVKVytleDlKeTFZOXEzUVpNZk0wWnFJbkxjOE43RFVnZnM4TktlUVh2SUFCNkxxdjBSNFY4VUNnYnZ6dEMKVitxdElGNUM5bmE5VFQzT3hVNkUwQnVmWTcwTmJBZ2dPN0RTN1FvQVc3ZG5HUnBDTmNieWg5dytZYi9vbkNCdgo3M28vRi9scnhFZ01jNFhuYTR6OGhXbm5STmdjcVBSVnNyWGFiVSt6TStsbVZEaEpwWE96dnVmMmZRb3creGF4CndaWnVwUmF5VDBESHVHbmpaQnkrNnFwQVdZampqaE9WOUhGcTlQQUpMUXAzR2xZdklueFgxUkJscTYyVFdZMW8KcjIvTFBRTUhjOUV6VFlVN21Qb0laQ3dqa1dPTzZmc1NFVHpBTk9ad1NlSlBRSW5XV1NlQXlsWjA4V2tNWjdVcQpDNkZHVml1REFVbE1HczBMMlhzQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFc1I5T2VSRW45NXdSaTA2UnI2SVhUcDhJeHkKSytmdFJyM1pZckZ5VWZZYTBWdU9Mc1NZdzByN1o5Zmk0ZUFlMEk4dnR2cWpqWWl6RzFnUFAyS3V4d0h4RmtJRApMQnlNRmRTSG5yVEVZeWo2NVFnbUtCWVpEZ0VkMnZpVnBTeHM4Y3dCZXgvT201VnErZnROanAwK2swaGdhV2xxCjBDZmtkbjM0MkY2bUhSZFNyeGg3eURleGNtNEtGck5OVnNPY0h5MEJhQXhwZ0JOUmErMG5oZ1dDbHh1M0F4OWgKaVRELzAyczRCcWZKTFZjZXk0Q2VnWW8zUDVDYWVjTTZSaTg4TVFKYlZ1OWx6RVZPUzlBRGNKZ0VkWkdHUFUyVwpEcHBhZXdZNjhSVVg1MG9ObXN2S2h1RGNCSWxyeHJ4T2J4Wk1wYm5hYldUMlFaKzM4ampEaDNwVmxhcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= | base64 -d > /tmp/ca.crt$ cat /tmp/ca.crt -----BEGIN CERTIFICATE-----MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJlcm5ldGVzMB4XDTE5MDYwNjEwMjYwOVoXDTI5MDYwMzEwMjYwOVowFTETMBEGA1UEAxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJPRagRUJW+ex9Jy1Y9q3QZMfM0ZqInLc8N7DUgfs8NKeQXvIAB6Lqv0R4V8UCgbvztCV+qtIF5C9na9TT3OxU6E0BufY70NbAggO7DS7QoAW7dnGRpCNcbyh9w+Yb/onCBv73o/F/lrxEgMc4Xna4z8hWnnRNgcqPRVsrXabU+zM+lmVDhJpXOzvuf2fQow+xaxwZZupRayT0DHuGnjZBy+6qpAWYjjjhOV9HFq9PAJLQp3GlYvInxX1RBlq62TWY1or2/LPQMHc9EzTYU7mPoIZCwjkWOO6fsSETzANOZwSeJPQInWWSeAylZ08WkMZ7UqC6FGViuDAUlMGs0L2XsCAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAEsR9OeREn95wRi06Rr6IXTp8IxyK+ftRr3ZYrFyUfYa0VuOLsSYw0r7Z9fi4eAe0I8vtvqjjYizG1gPP2KuxwHxFkIDLByMFdSHnrTEYyj65QgmKBYZDgEd2viVpSxs8cwBex/Om5Vq+ftNjp0+k0hgaWlq0Cfkdn342F6mHRdSrxh7yDexcm4KFrNNVsOcHy0BaAxpgBNRa+0nhgWClxu3Ax9hiTD/02s4BqfJLVcey4CegYo3P5CaecM6Ri88MQJbVu9lzEVOS9ADcJgEdZGGPU2WDppaewY68RUX50oNmsvKhuDcBIlrxrxObxZMpbnabWT2QZ+38jjDh3pVlas=-----END CERTIFICATE-----

获取/root/.kube/config中client-certificate-data和client-key-data的内容并转化成base64 encoded文件

$ echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJSzZSZWRVQ2NNS0V3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RBMk1EWXhNREkyTURsYUZ3MHlNREEyTURVeE1ESTJNVEZhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTBrcW1jci9DNHBaSTgwUTAKTzBOaklRV1hPelQ0TmVPVFJQd0N1UW1CaDlZcUJ0QTBkZnJvVnRzT2JRTEdlWE5GUVlhRzM5V2ZsN292YURCNwp3Wm11WVo3alZKUm9BZmpldEg3d0lIN3pNYUdCbk1hN3RoTUNqWXBmdXRpQyszQm51cjFLRmFJR2FKcDNlK0NQCjAyVjdSVDJjUXFPbHVOVUtFSEl2UG15YTdKM0pHWUpLNFVBVEhQOVhIMVZNemVoc3N5ZjF5UTZZb1BxTTR1b08KVGVNN3g0NFcvS2hrWjBMWkhncnB5RDJQNlN2NGVoaGl6YTQrdUk1Si80OFhhVFJwMUxINmdGODVhalkwNXIybApQUW95dDFPTEdIVHZtRnA3ei9QMHBKbUJpcGQwcktBeFNJLzU5SU93elYyRFNpQmp4TjQwSWpKNWhidlczNHNFCmtHNTFTUUlEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFWGM0VUhiOVFMSisvU2t0S3dhOVpydVJ3THBhV0NVeE80Sgo1Mml0eHdZK3JLdE9WQVBsKzdNRjVmcWU3UUhmcW0yMjFmR1V6UW50VERpM1RjOVNsdUVDdjdxdHBCcHczZ1pYCnZpSmIvakRRTWwrdmhvZW95UlpiUmhWZG5kUU5lS1FoUzl4UEw3Rmx2d0pRUnNsWTlCdWJ4L3lsVC9KRElKcFcKRzJhanM4ZGNyWUlLeGlEbjdQSTZqeFB1OURRVUgraGVza1M5VXRwcUF3M3pBdjFCaG5kV2FEWUNQc293MnloVAp1RFNrWG9XVE9rTlQ2UDVWREdob3VZbzJ4emhJeDl4NWllMVpLOHlnODY5THZvZkJQNk5oRnZRNDRPQkFyR29JCnNJcDNJT0ZpaWJjVW1wVXNZNWJ2OEVXbjBNdXkxN1VjVTlvMkYvTnpuQ1hZV1JEVUViVT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= | base64 -d > /tmp/client.crt$ echo LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMGtxbWNyL0M0cFpJODBRME8wTmpJUVdYT3pUNE5lT1RSUHdDdVFtQmg5WXFCdEEwCmRmcm9WdHNPYlFMR2VYTkZRWWFHMzlXZmw3b3ZhREI3d1ptdVlaN2pWSlJvQWZqZXRIN3dJSDd6TWFHQm5NYTcKdGhNQ2pZcGZ1dGlDKzNCbnVyMUtGYUlHYUpwM2UrQ1AwMlY3UlQyY1FxT2x1TlVLRUhJdlBteWE3SjNKR1lKSwo0VUFUSFA5WEgxVk16ZWhzc3lmMXlRNllvUHFNNHVvT1RlTTd4NDRXL0toa1owTFpIZ3JweUQyUDZTdjRlaGhpCnphNCt1STVKLzQ4WGFUUnAxTEg2Z0Y4NWFqWTA1cjJsUFFveXQxT0xHSFR2bUZwN3ovUDBwSm1CaXBkMHJLQXgKU0kvNTlJT3d6VjJEU2lCanhONDBJako1aGJ2VzM0c0VrRzUxU1FJREFRQUJBb0lCQUE3bXcweTJVZlVFZVQ3agp3bC9Bc3JHUVY5c1dNZEIvdzl2TGo5WFUycHpwakNqWGNDQThHMktzT3lWMllPSVNUUUlMcWxzS0pEajROSXZKCmc3dUFURjhXaHoxakZzdXMrdnNIVTdTNXlqbm1HKzBrR0FFYTc3OWY0dEMycnZGcVVhOWw0bTRPQVM1QVk5OGYKVnBIQVN5L280YjNISXVNcUZZQjgxdVF4aGZqbVJHTWJ3dFdRM090dUU1d2xQRDJwOFBqTElaQVNsVTZ2RlpoQQpMWCs0ZjIyMWRjSkxTVGhqTDFiVHFPM1NkazE3WjZOV1M1Ym9TMFptY0JSOFhLUHNqUHFVek5zRFVxeCs1cDdKCkFXVGNlYjB4dWZYTldDV3Y3N3NUcTBjdyt6UXNwbVZwbFkzMjA4d0c5Tm9FYm5hMGFNQ1ZacG1oK3BSR2xOelkKRlNZNU9rRUNnWUVBMGxLcjI2S3IvMU1KcGFGMHRtWWNHcVRzd093bklBMzBNemk2blN5SVd1Q21qRWs3ZU5zSgpTRUk5cWRPRzhhWGw3VGZpa2VYUko1b3MxcDFZZzh2TUdpam01NGI3ejYybFZRUWtsOWZPSlVGM1YzQkgyR2p2CjRpOFZVTUl5ZmNSRmZUUFZEeGtKbHZtdnZCS0kwZU9UV0daVmFxb1AzZ0d0SW5lVDlmbEgwUTBDZ1lFQS8vWTgKbzZaelRtejIyVWVXZW5aemlNOHBxRHBqVU5DRnlKemJqajNvN1ZsTGRSc0FybnpFZnVNaUo5YjZ2dVpaWXQzRwp2bEZFMDBiYnNQWmRVbzArenJvSEZMK2JqQ1QrODl1UUFJRHFHcGtyYUdIZ0IxelZyMHF2L0NXajlnbzkrdUxhCjJKQk5ud1RSczlsbzE1ZzV1YnJPRWFBNzlBaExEeEFuZFA3Z0RpMENnWUFyODBpa2NmN2RNUDRBRlpndEVYTm8KQWZUVGI4WFJSZmsweHZNQUt6RW5SSENwT2hocWJlTW5yV2Z6V0JlSDRiSUZlenNtWDg3d0pxQ2VERzFWeFQyVwpiZHVxb0NONHg0R1lIWENFSm4yV2ZYS3gyKzIzaEY0MGRzQk9pdlpBSDhhaG5qWTBuSGZMaTh1MFVtOHk1UXFDClc0Z1g3UWU1emNIZlJQdXZWL013OVFLQmdRQy8rKzFYd2cxU2thQkZNTkRKWWZjZWNtUUlibUwzeHEvUjNQVkIKSjJhQ1FDdTgxbGdZaURUS0I0c2kzcmlNWHpKRVdad3NPOENueDhvWVhYRjU3YjlpUjEzd1RoZFpjcFpZU2lNawpmWTBhRGpEa3hpVEc0UGJWMSt0UDhOdWVPK2hwT2FaME1TaEhVZElJVjlXdmY5b3NXTlVmbTFQY29pdktUSStMCnpYQTRzUUtCZ0ZsUnpoeDlYczJ2L1h1VGl0NE1mNUpMU2ppNk93c3pZVXNmTzVMQ1BVbGQrcjA5dmFsS0tPQlcKU1FUVDhTdnBpMzZhQVFTV0xCUUdVYkh2VlJ0QkJHOWxiY2JSSHlla1RrYksxTEwxVk0zKzdtRXZCUDFZZU9qZQpwWkRlTWcrZWQyY3F5MDJhZWwxbGhBK3dJRmlnUzV5bGY0Z25paFBnMVZ2WmJIUGIzREdGCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== | base64 -d > /tmp/client.key

将上面生成的文件转换为P12认证文件cert.pfx，并下载至本地；生成过程中设置的密码要记住，后面有用

$ openssl pkcs12 -export -out /tmp/cert.pfx -inkey /tmp/client.key -in /tmp/client.crt -certfile /tmp/ca.crtEnter Export Password:Verifying - Enter Export Password: $ sz /tmp/cert.pfx

然后回到jenkins配置全局凭据

最终新增kubernetes云配置，点击连接测试，提示成功即可

配置第二个k8s相关信息，过程和配置第一个k8s一样，不再过多说明了

创建流水线项目

打开Bule Ocean，请创建你的第一个流水线，选择代码仓库为Git

输入要创建流水线的Git项目仓库地址，输入后jenkins会自动生成公钥，把生成的公钥配置在gitlab的ssh key中，然后点击创建流水线

创建流水线时候，Jenkins会自动检测git项目各个分支的根目录是否存在文件“Jenkinsfile”，如果存在就生成一个分支流水线，下图中生成了分支master和分支docker流水线

到目前为止流水线已经配置完成，但是还无法实现自动触发构建，需要配置扫描多分支流水线触发器；设置1分钟检测一次

触发构建之前我们先看下流水线执行构建部署的Jenkinsfile文件内容，文件保存在git项目的各个分支

def label = "slave-$UUID.randomUUID().toString()"podTemplate(cloud: 'kubernetes', label: label, containers: [ containerTemplate(name: 'docker', image: 'docker', command: 'cat', ttyEnabled: true), containerTemplate(name: 'kubectl', image: 'bitnami/kubectl', command: 'cat', ttyEnabled: true),], volumes: [ hostPathVolume(mountPath: '/root/.kube', hostPath: '/root/.kube'), hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')])  node(label)  def myRepo = checkout scm def gitCommit = myRepo.GIT_COMMIT def gitBranch = myRepo.GIT_BRANCH def imageTag = sh(script: "git rev-parse --short HEAD", returnStdout: true).trim() def dockerRegistryUrl = "registry.cn-beijing.aliyuncs.com" def imageEndpoint = "addnewer-dsc/approval-fe" def image = "$dockerRegistryUrl/$imageEndpoint" stage('构建 Docker 镜像')  withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'DockerRegistry', usernameVariable: 'DOCKER_HUB_USER', passwordVariable: 'DOCKER_HUB_PASSWORD']])  container('docker')  echo "3. 构建 Docker 镜像阶段" sh """ docker login $dockerRegistryUrl -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD docker build -t $image:$imageTag . docker push $image:$imageTag """    stage('Run kubectl')  container('kubectl')  sh """ sed -i "s#<IMAGE>#$image#g" *.yaml sed -i "s#<IMAGE_TAG>#$imageTag#g" *.yaml kubectl apply -f . """

podTemplate(cloud: 'kubernetes' 我们要实现不同分支部署到不同k8s环境，所以我们不同分支中Jenkinsfile中pod模版要指定不同的cloud，这个很重要；默认名称为kubernetes，这个名称对应我们在jenkins中新增k8s云的名称containerTemplate(name: 'docker' 指定我们需要构建镜像需要用到的docker镜像containerTemplate(name: 'kubectl' 指定我们部署容器到k8s需要用到的kubectl镜像hostPathVolume(mountPath: 上面指定镜像依赖的映射文件def imageTag 生成镜像tag名称def dockerRegistryUrl 定义docker仓库地址stage('构建 Docker 镜像') 定义流水线构建docker镜像执行步骤 credentialsId: 'DockerRegistry', 从jenkins全局凭据获取docker仓库id usernameVariable: 'DOCKER_HUB_USER',从jenkins全局凭据获取docker仓库用户名 passwordVariable: 'DOCKER_HUB_PASSWORD'从jenkins全局凭据获取docker仓库密码stage('Run kubectl') 定义流水线部署应用到k8s执行步骤，部署应用的yaml文件也是在git项目中后缀为yaml的文件

了解了上面的Jenkinsfile流水线文件的执行流程后，可以看到我们还没有在jenkins中添加docker仓库的全局凭据，下面我们添加

流水线构建测试

提交任何变更到git项目，1分钟后jenkins会自动检测到变更，开始执行流水线；这里我随便提交下测试代码，就可以看到流水开始执行了