基于jenkins+gitlab+springcloud+harbor+nexus构建Devops

1、环境准备(k8s集群)

1
2
3
4
5
6
[root@master1 ~]# kubectl get nodes
NAME      STATUS   ROLES           AGE    VERSION
master1   Ready    control-plane   112d   v1.25.0
master2   Ready    control-plane   108d   v1.25.0
node1     Ready    work            112d   v1.25.0
node2     Ready    work            109d   v1.25.0

Sonarqube部署机器ip：192.168.101.12

nexus部署机器ip：192.168.101.12

gitlab部署机器ip：192.168.101.12

harbor部署机器ip：35.220.201.102

2、Jenkins安装

Jenkins安装的是2.379版本；

Kubernetes集群版本是1.25.0

2.1 配置nfs服务（node1上）

1
2
3
4
5
6
[root@node1 ~]# exportfs -arv
exporting *:/nfs-data/v2
exporting *:/nfs-data/v1
[root@node1 ~]# cat /etc/exports
/nfs-data/v1 *(rw,no_root_squash)
/nfs-data/v2 *(rw,no_root_squash)

2.2 制作Jenkins 2.379镜像

1
2
3
4
5
6
7
8
9
10
11
12
[root@node1 dockerfile]# ll
total 96072
-rw-r--r-- 1 root root      125 Mar 30 13:54 dockerfile
-rw-r--r-- 1 root root 98370482 Mar 30 13:48 jenkins.war
[root@node1 dockerfile]# cat dockerfile 
FROM jenkins/jenkins:latest
MAINTAINER zy
#RUN rm -rf /usr/share/jenkins/jenkins.war
COPY ./jenkins.war /usr/share/jenkins/ 


docker build -f dockerfile -t "jenkins/jenkins:2.379" .

由于1.25.0采用的是containerd来管理的容器,制作好的镜像要在docker中导出保存，然后导入containerd中

1
2
3
4
5
6
7
8
9
10
11
[root@node1 ~]# docker images
REPOSITORY        TAG         IMAGE ID       CREATED             SIZE
jenkins/jenkins   2.379       3058c5c258e0   About an hour ago   541MB


#导出docker镜像
[root@node1 ~]# docker save -o jenkins-2.379.tar jenkins/jenkins:2.379 


#导入containerd中
[root@node1 ~]# ctr -n k8s.io images import jenkins-2.379.tar

2.3 创建挂载卷(pv与pvc)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@master1 jenkins_gitlab_springcloud_harbor_nexus_k8s_Devops]# cat pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata: 
  name: jenkins-k8s-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
  - ReadWriteMany
  nfs:
    server: 192.168.101.21
    path: /nfs-data/v2
    
    
    
[root@master1 jenkins_gitlab_springcloud_harbor_nexus_k8s_Devops]# cat pvc.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata: 
  name: jenkins-k8s-pvc
  namespace: jenkins-k8s
spec:
  resources:
    requests:
      storage: 10Gi
  accessModes:
  - ReadWriteMany

2.4 创建sa账号并作授权(master节点)

1
2
3
kubectl create ns jenkins-k8s
kubectl create sa jenkins-k8s-sa -n jenkins-k8s
kubectl create clusterrolebinding jenkins-k8s-sa-cluster -n jenkins-k8s  --clusterrole=cluster-admin --serviceaccount=jenkins-k8s:jenkins-k8s-sa

3.5 部署Jenkins

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
[root@master1 jenkins_gitlab_springcloud_harbor_nexus_k8s_Devops]# cat jenkins-deployment.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: jenkins
  namespace: jenkins-k8s
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      serviceAccount: jenkins-k8s-sa
      containers:
      - name: jenkins
        image:  jenkins/jenkins:2.379
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        - containerPort: 50000
          name: agent
          protocol: TCP
        resources:
          limits:
            cpu: 2000m
            memory: 2Gi
          requests:
            cpu: 500m
            memory: 512Mi
        livenessProbe:
          httpGet:
            path: /login
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12
        readinessProbe:
          httpGet:
            path: /login
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12
        volumeMounts:
        - name: jenkins-volume
          subPath: jenkins-home
          mountPath: /var/jenkins_home
      volumes:
      - name: jenkins-volume
        persistentVolumeClaim:
          claimName: jenkins-k8s-pvc

3.6 解决出现的报错（node1上）

1
chown -R 1000.1000 /nfs-data/v2

3.7 配置svc四层代理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@master1 jenkins_gitlab_springcloud_harbor_nexus_k8s_Devops]# cat jenkins_svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
  namespace: jenkins-k8s
  labels:
    app: jenkins
spec:
  selector:
    app: jenkins
  type: NodePort
  ports:
  - name: web
    port: 8080
    targetPort: web
    nodePort: 30002
  - name: agent
    port: 50000
    targetPort: agent
1
2
3
4
5
6
7
8
[root@master1 ~]# kubectl get pods -n jenkins-k8s
NAME                       READY   STATUS    RESTARTS   AGE
jenkins-5d4bcc79c4-qvmsv   1/1     Running   0          70m


[root@master1 ~]# kubectl get svc -n jenkins-k8s    
NAME              TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                          AGE
jenkins-service   NodePort   10.255.131.109   <none>        8080:30002/TCP,50000:31377/TCP   69m

3、Jenkins的配置

3.1 安装插件

kubernetes插件

image-20230330151234922

blueocean插件

image-20230330151344113

注意：安装完插件需要重启才能生效

3.2 配置连接kebernetes集群

访问http://192.168.101.11:30002/configureClouds/

image-20230330151713936

image-20230330151740295

image-20230330151815421

image-20230330151850084

image-20230330152005553

image-20230330152030328

image-20230330152041622

3.3 添加dockerhub凭据

image-20230330153055128

3.4 拷贝.kube文件到node节点上

1
2
[root@master1 ~]# scp -r .kube 192.168.101.21:~
[root@master1 ~]# scp -r .kube 192.168.101.22:~

4、测试通过Jenkins部署应用发布到k8s开发、生产、测试环境

开发提交代码到代码仓库gitlab—->jenkins检测到代码更新—–>调用k8s api在k8s中创建jenkins slave pod：

Jenkins slave pod拉取代码—–>通过maven把拉取的代码进行构建成war包或者jar包—>上传代码到Sonarqube，进行静态代码扫描- –>基于war包构建docker image–>把镜像上传到harbor镜像仓库–>基于镜像部署应用到开发环境–>部署应用到测试环境—>部署应用到生产环境。

image-20230331104405060

4.1 创建所需名称空间

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@master1 ~]# kubectl create ns devlopment
namespace/devlopment created
[root@master1 ~]# kubectl create ns production
namespace/production created
[root@master1 ~]# kubectl create ns qatest
namespace/qatest created


[root@master1 ~]# kubectl get ns
NAME              STATUS   AGE
default           Active   113d
devlopment        Active   28s
jenkins-k8s       Active   43h
kube-node-lease   Active   113d
kube-public       Active   113d
kube-system       Active   113d
production        Active   18s
qatest            Active   12s

4.2 制作Jenkins-agent镜像

注意：要在能科学上网的节点上进行操作

由于从官方下载的Jenkins-agent中无docker、kubectl命令，且其中的/var/run/docker.sock

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
[root@harbor dockerfile]# cat dockerfile 
FROM jenkins/inbound-agent:latest
ARG HOST_GID=991
USER root
USER root
RUN apt-get -y update && \
    apt-get -y install apt-transport-https ca-certificates curl gnupg-agent software-properties-common && \
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
    add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable" && \
    apt-get update && \
    apt-get -y install docker-ce docker-ce-cli containerd.io
RUN curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose  \
    && chmod +x /usr/local/bin/docker-compose  \
    && ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose \
    && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - 
RUN echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list \
    && apt-get -y update \
    && apt install -y kubectl

RUN groupmod -g $HOST_GID docker
RUN usermod -aG docker jenkins



[root@harbor dockerfile]# docker build -f dockerfile  -t "jenkins/jnlp:v1" . 
[+] Building 15.3s (10/10) FINISHED                                                                                 
 => [internal] load build definition from dockerfile                                                           0.0s
 => => transferring dockerfile: 1.14kB                                                                         0.0s
 => [internal] load .dockerignore                                                                              0.0s
 => => transferring context: 2B                                                                                0.0s
 => [internal] load metadata for docker.io/jenkins/inbound-agent:latest                                        0.0s
 => [1/6] FROM docker.io/jenkins/inbound-agent:latest                                                          0.0s
 => CACHED [2/6] RUN apt-get -y update &&     apt-get -y install apt-transport-https ca-certificates curl gnu  0.0s
 => CACHED [3/6] RUN curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(una  0.0s
 => [4/6] RUN echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kube  6.0s
 => [5/6] RUN groupmod -g 991 docker                                                                           0.7s
 => [6/6] RUN usermod -aG docker jenkins                                                                       0.5s 
 => exporting to image                                                                                         8.1s 
 => => exporting layers                                                                                        8.1s 
 => => writing image sha256:81bfceafc0e69eaa425523f4b8749a8b31eb0041afd14991efe38935f5182f20                   0.0s 
 => => naming to docker.io/jenkins/jnlp:v1                                                                     0.0s 
 
 
 
 #将镜像导入node节点中
 [root@node1 ~]# ctr -n k8s.io image import jenkins-jnlp-v1.tar 
unpacking docker.io/jenkins/jnlp:v1 (sha256:659a7d21499b44da44f6fc72bd40bbed36a527b19d37cff75c51037632434fcb)...done

4.3 新建pipeline脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
node('test') {
    stage('Clone') {
        echo "1.Clone Stage"
        git url: "https://kgithub.com/vorabend/jenkins-sample.git"
        script {
            build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
        }
    }
    stage('Test') {
      echo "2.Test Stage"

    }
    stage('Build') {
        echo "3.Build Docker Image Stage"
        sh "docker build -t vorabend/jenkins-demo:${build_tag} ."
    }
    stage('Push') {
        echo "4.Push Docker Image Stage"
        withCredentials([usernamePassword(credentialsId: 'dockerhub', passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
            sh "docker login -u ${dockerHubUser} -p ${dockerHubPassword}"
            sh "docker push vorabend/jenkins-demo:${build_tag}"
        }
    }
    stage('Deploy to dev') {
        echo "5. Deploy DEV"
		sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s-dev.yaml"
        sh "sed -i 's/<BRANCH_NAME>/${env.BRANCH_NAME}/' k8s-dev.yaml"
//        sh "bash running-devlopment.sh"
        sh "kubectl apply -f k8s-dev.yaml  --validate=false"
	}	
	stage('Promote to qa') {	
		def userInput = input(
            id: 'userInput',

            message: 'Promote to qa?',
            parameters: [
                [
                    $class: 'ChoiceParameterDefinition',
                    choices: "YES\nNO",
                    name: 'Env'
                ]
            ]
        )
        echo "This is a deploy step to ${userInput}"
        if (userInput == "YES") {
            sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s-qa.yaml"
            sh "sed -i 's/<BRANCH_NAME>/${env.BRANCH_NAME}/' k8s-qa.yaml"
//            sh "bash running-qa.sh"
            sh "kubectl apply -f k8s-qa.yaml --validate=false"
            sh "sleep 6"
            sh "kubectl get pods -n qatest"
        } else {
            //exit
        }
    }
	stage('Promote to pro') {	
		def userInput = input(

            id: 'userInput',
            message: 'Promote to pro?',
            parameters: [
                [
                    $class: 'ChoiceParameterDefinition',
                    choices: "YES\nNO",
                    name: 'Env'
                ]
            ]
        )
        echo "This is a deploy step to ${userInput}"
        if (userInput == "YES") {
            sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s-prod.yaml"
            sh "sed -i 's/<BRANCH_NAME>/${env.BRANCH_NAME}/' k8s-prod.yaml"
//            sh "bash running-production.sh"
            sh "cat k8s-prod.yaml"
            sh "kubectl apply -f k8s-prod.yaml --record --validate=false"
        }
    }
}

4.4 运行pipeline脚本

全部运行成功

image-20230401232331273

通过上面Jenkins对接K8S，就可以把应用发送到K8S集群的开发、测试、生产环境中去了

1
2
3
4
5
6
[root@master1 ~]# kubectl get pods -n qatest
NAME                            READY   STATUS    RESTARTS   AGE
jenkins-demo-68dd658bc7-qtjs5   1/1     Running   0          15h
[root@master1 ~]# kubectl get pods -n production 
NAME                            READY   STATUS    RESTARTS   AGE
jenkins-demo-68dd658bc7-blgjv   1/1     Running   0          15h

5、配置拉取harbor镜像仓库

由于采用的k8s版本是1.25.0，此版本k8s已经不使用docker作为容器运行时，而是采用containerd作为容器运行时，但是一般情况下我们docker也需要安装，因为需要制作镜像。

5.1 修改配置文件（config.toml与hosts）

master和node所有节点上修改/etc/containerd文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
cd /etc/containerd


# 编辑containerd的配置文件
vim config.toml


#修改过的地方
 [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = ""
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
          endpoint = ["https://registry.cn-hangzhou.aliyuncs.com"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."35.220.201.102"]
          endpoint = ["http://35.220.201.102"]
      [plugins."io.containerd.grpc.v1.cri".registry.configs]
        [plugins."io.containerd.grpc.v1.cri".registry.configs."35.220.201.102".tls]
           insecure_skip_verify = true
        [plugins."io.containerd.grpc.v1.cri".registry.configs."35.220.201.102".auth]
           username = "admin"
           password = "harbor的密码"
           
           
           
#修改主机的hosts文件
[root@master1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.101.11 master1
192.168.101.12 master2
192.168.101.21 node1
192.168.101.22 node2
35.220.201.102 harbor

5.2 测试拉取镜像是否成功

1
2
[root@master1 ~]# crictl pull 35.220.201.102/microservice/eureka:v1
Image is up to date for sha256:65d9615ad919f1704881006d8de94f67e23e93173ef0edb97bad0953d72ef49b

6、安装sonarqube

SonarQube 是一个开源的代码分析平台, 用来持续分析和评测项目源代码的质量。 通过SonarQube我们可以检测出项目中重复代码， 潜在bug， 代码规范，安全性漏洞等问题， 并通过SonarQube web UI展示出来。

6.1 调整vm.max_map_count参数的大小

max_map_count文件包含限制一个进程可以拥有的VMA(虚拟内存区域)的数量

1
2
3
4
5
6
7
8
9
10
[root@master2 ~]# sysctl -a | grep vm.max_map_count
vm.max_map_count = 65530


[root@master2 ~]# cat  /etc/sysctl.conf  | grep -v ^#
vm.max_map_count=262144


[root@master2 ~]# sysctl -p
vm.max_map_count = 262144

6.2 上传与解压postgres.tar.gz sonarqube.tar.gz镜像文件

1
2
3
4
5
[eve-ng 课程资料] $ scp postgres.tar.gz sonarqube.tar.gz 192.168.101.12:~


[root@master2 ~]# docker load -i postgres.tar.gz
[root@master2 ~]# docker load -i sonarqube.tar.gz

6.3 运行容器

1
2
3
4
5
[root@master2 ~]# docker run  -d   --name postgres10   -p 5432:5432   -e POSTGRES_USER=sonar   -e POSTGRES_PASSWORD=123456   postgres


[root@master2 ~]# docker run   -d  --name sonarqube7.9   -p 9000:9000   --link postgres10   -e SONARQUBE_JDBC_URL=jdbc:postgresql://postgres10:5432/sonar   -e  SONARQUBE_JDBC_USERNAME=sonar   -e SONARQUBE_JDBC_PASSWORD=123456   -v sonarqube_conf:/opt/sonarqube/conf   -v sonarqube_extensions:/opt/sonarqube/extensions   -v sonarqube_logs:/opt/sonarqube/logs   -v sonarqube_data:/opt/sonarqube/data   sonarqube

6.4 浏览器访问sonarqube

默认的账号密码都是admin

image-20230407215752536

6.5 在jenkins中安装sonarqube插件

在jenkins中安装sonarqube插件：

系统管理->插件管理->可选插件：搜索sonar，找到Sonarqube Scanner->选择Sonarqube Scanner直接安装，安装之后重启jenkins即可:

image-20230407220110144

6.6 在sonarqube的web界面上创建一个token

image-20230407220319143

image-20230407220430585

1
2
#生成的token
4158c00d423a6ef54f37d22e2fb2dfad4ca31ab1

6.5 将源码包上传到控制节点编译打包

1
2
3
4
5
[root@master1 jenkins_gitlab_springcloud_harbor_nexus_k8s_Devops]# unzip microservic-test.zip


[root@master1 microservic-test]# gg yum install maven -y
[root@master1 microservic-test]# mvn sonar:sonar  -Dsonar.host.url=http://192.168.101.12:9000 -Dsonar.login= 4158c00d423a6ef54f37d22e2fb2dfad4ca31ab1

7、Jenkins界面添加harbor凭据

image-20230407223123749

8、安装nexus

概念：

Nexus服务器是一个代码包管理的服务器，可以理解 Nexus 服务器是一个巨大的 Library 仓库。Nexus 可以支持管理的工具包括 Maven ， npm 等，对于 JAVA 开发来说，只要用到 Maven 管理就可以了。

Nexus服务器作用：因为传统的中央仓库在国外，其地理位置比较远，下载速度比较缓慢。因此，当公司开发人员数量越来越多时，如果不架设一台自己的Nexus服务器，会产生大量的流量阻塞带宽，并且在出现一些不可抗原因（光缆被挖断）导致无法连接到中央仓库时，开发就会因为无法下载相关依赖包而进度停滞。因此在本地环境部署一台私有的Nexus服务器来缓存所有依赖包，并且将公司内部开发的私有包也部署上去，方便其他开发人员下载，是非常有必要的。因为 Nexus 有权限控制，因此外部人员是无法得到公司内部开发的项目包的。

8.1 解压镜像启动容器

1
2
3
4
5
6
7
8
9
[root@master2 ~]# docker load -i nexus3.tar.gz 
Loaded image: sonatype/nexus3:latest


[root@master2 ~]# docker run -d -p 8081:8081 -p 8082:8082 -p 8083:8083 -v /etc/localtime:/etc/localtime --name nexus3   sonatype/nexus3

#在日志中，您会看到一条消息： Started Sonatype Nexus OSS 3.20.1-01 这意味着您的Nexus Repository Manager可以使用了。现在转到浏览器并打开http://your-ip-addr:8081
[root@master2 ~]# docker logs 79d90bf3e728 | grep -i ^started
Started Sonatype Nexus OSS 3.37.3-02

image-20230407224504825

8.2 使用nexus

第一步：

1、在 pom.xml 文件中声明发布的宿主仓库和 release 版本发布的仓库。

1
2
3
4
5
6
7
8
9
10
11
12
13
<!-- 发布构件到Nexus -->
    <distributionManagement>
        <repository>
            <id>releases</id>
            <name>nexus-releases</name>
            <url>http://192.168.40.135:8081/repository/maven-releases/</url>
        </repository>
        <snapshotRepository>
            <id>snapshots</id>
            <name>nexus-snapshots</name>
            <url>http://192.168.101.12:8081/repository/maven-snapshots/</url>
        </snapshotRepository>
    </distributionManagement>

第二步：在 settings.xml 文件中配置
由于用 Maven 分发构件到远程仓库需要认证，须要在~/.m2/settings.xml或者中加入验证信息：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<servers>  
   <server>  
           <id>public</id>  
           <username>admin</username>  
           <password>123456</password>  
       </server>  
   <server>  
           <id>releases</id>  
           <username>admin</username>  
           <password>123456</password>  
       </server>  
   <server>  
           <id>snapshots</id>  
           <username>admin</username>  
           <password>123456</password>  
       </server>  
 </servers>

注意： settings.xml 中 server 元素下 id 的值必须与 POM 中 repository 或 snapshotRepository 下 id 的值完全一致 。　

9、安装gitlab

9.1 导入镜像启动容器

1
2
3
4
5
6
[root@master2 ~]# docker load -i gitlab.tar.gz 



[root@master2 ~]# docker run -d  -p 443:443 -p 80:80 -p 222:22 --name gitlab --restart always -v /home/gitlab/config:/etc/gitlab -v /home/gitlab/logs:/var/log/gitlab -v /home/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce 
ad6905342c658fde1fa570b81b36569a9e41d08c6b208c0c1eeba34842d7f566

9.2 修改gitlab的配置文件重启docker

1
2
3
4
5
6
7
[root@master2 config]# cat gitlab.rb | grep ^external_url -A3
external_url = 'http://192.168.101.12'
gitlab_rails['gitlab_ssh_host'] = '192.168.101.12'
gitlab_rails['gitlab_shell_ssh_port'] = 222


[root@master2 config]# docker restart gitlab

9.3 进入gitlab容器中修改账号密码

登录账号：root

登陆密码：12345678

确保Jenkins中安装了Gitlab插件

1
2
3
4
5
6
[root@master2 ~]# docker exec -it gitlab /bin/bash
root@ad6905342c65:/# gitlab-rails console
u=User.where(id:1).first
u.password='12345678'
u.password_confirmation='12345678' 
u.save!

9.4 Jenkins界面添加gitlab凭据

image-20230408131925038

9.5 登录gitlab新建项目上传代码

image-20230408144849075

1
2
3
4
5
6
7
8
git config --global user.email "[email protected]"
git config --global user.name "ZhangYu"
git remote add origin http://192.168.101.12/root/microservice-test.git
[root@master1 microservic-test]# git push -u origin --all
Username for 'http://192.168.101.12': root
Password for 'http://[email protected]': 
branch 'master' set up to track 'origin/master'.
Everything up-to-date

image-20230408144947811

10、构建Devops

10.1 在harbor仓库上创建一个项目（microservice）

image-20230408153721252

10.2 创建拉取harbor私有仓库镜像的secert

1
[root@master1 ~]# kubectl create ns ms && kubectl create secret docker-registry registry-pull-secret --docker-server=35.220.201.102 --docker-username=admin --docker-password=密码  -n ms

10.3 导入docker.io/jenkins/jnlp:v2镜像（node节点）

通过dockerfile构建镜像，里面封装了docker、kubectl、maven.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[root@harbor dockerfile]# cat dockerfile 
FROM jenkins/inbound-agent:latest
ARG HOST_GID=991
USER root
USER root
RUN apt-get -y update && \
    apt-get -y install apt-transport-https ca-certificates curl gnupg-agent software-properties-common && \
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
    add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable" && \
    apt-get update && \
    apt-get -y install docker-ce docker-ce-cli containerd.io
RUN curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose  \
    && chmod +x /usr/local/bin/docker-compose  \
    && ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose \
    && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - 
RUN echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list \
    && apt-get -y update \
    && apt install -y kubectl

RUN groupmod -g $HOST_GID docker
RUN usermod -aG docker jenkins
RUN apt-get install maven -y



[root@harbor dockerfile]# docker build -f dockerfile -t "jenkins/jnlp:v2" . 


[root@node1 ~]# ctr -n k8s.io images import jnlp-v2.tar
[root@node2 ~]# ctr -n k8s.io images import jnlp-v2.tar


[root@node2 ~]# ctr -n k8s.io image ls | grep jnlp
docker.io/jenkins/jnlp:v2                                                                                                                      application/vnd.docker.distribution.manifest.v2+json      sha256:28cdf386fa72816b9421a87cc514cb671a7942111bc041bc64740c8806fd4d2c 1.1 GiB   linux/amd64                                                                                                                        io.cri-containerd.image=managed

10.4 更改jenkins模版的镜像为docker.io/jenkins/jnlp:v2

image-20230409184027766

10.5 修改gitlab上的portal.yaml

把image镜像变成：35.220.201.102/microservice/jenkins-demo:v1

10.6 编写pipeline脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
node('test') {
   stage('Clone') {
       echo "1.Clone Stage"
      git credentialsId: 'gitlab', url: 'http://192.168.101.12/root/microservice-test.git'
       script {
           build_tag = sh(returnStdout: true, script: 'git rev-parse --shortHEAD').trim()
       }
    }

   stage('Test') {
     echo "2.Test Stage"
    }
   stage('mvn') {
     sh "mvn clean package -D maven.test.skip=true"
    }
   stage('Build') {
       echo "3.Build Docker Image Stage"
       sh "cd /home/jenkins/agent/workspace/mvn-gitlab-harbor-springcloud/portal-service"
       sh "docker build --tag 35.220.201.102/microservice/jenkins-demo:v1 /home/jenkins/agent/workspace/mvn-gitlab-harbor-springcloud/portal-service/"
    }
   stage('Push') {
       echo "4.Push Docker Image Stage"
       withCredentials([usernamePassword(credentialsId: 'harbor',passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
           sh "docker login 35.220.201.102 -u ${dockerHubUser} -p ${dockerHubPassword}"
           sh "docker push 35.220.201.102/microservice/jenkins-demo:v1"
       }
    }
     stage('Promoteto pro') {    
    sh "kubectl apply -f /home/jenkins/agent/workspace/mvn-gitlab-harbor-springcloud/k8s/portal.yaml"
      }
}

image-20230409195144363

CKS
云计算 k8s

本博客所有文章除特别声明外，均采用 CC BY-SA 4.0 协议 ，转载请注明出处！