02.基于K8S部署Gitlab版本控制(helm方式部署)

集群节点ip
k8s-master-01192.168.1.2014C8G 1T控制节点、存储节点
k8s-worker-01192.168.1.20212C48G 200G工作节点
k8s-worker-02192.168.1.20312C48G 200G工作节点

1. 部署准备

  • 本次部署是基于 vm 虚拟机的测试环境,一主 2 从
  • 必须安装好ingress-nginx
  • 必须有默认的 storage
[admin@k8s-master-01 kubernetes]$ kubectl get all -n ingress-nginx 
NAME                                 READY   STATUS    RESTARTS     AGE
pod/ingress-nginx-controller-9v9hf   1/1     Running   6 (2d ago)   3d3h
pod/ingress-nginx-controller-f9m4v   1/1     Running   6 (2d ago)   3d3h

NAME                                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.109.140.92   <none>        80:30080/TCP,443:30443/TCP   3d3h
service/ingress-nginx-controller-admission   ClusterIP   10.99.115.202   <none>        443/TCP                      3d3h

NAME                                      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/ingress-nginx-controller   2         2         2       2            2           kubernetes.io/os=linux   3d3h

[admin@k8s-master-01 kubernetes]$ kubectl get sc
NAME                    PROVISIONER       RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-storage (default)   nfs-provisioner   Delete          Immediate           false                  3d3h
1.1 host 文件解析
Windows系统配置:C:\Windows\System32\drivers\etc\hosts
192.168.1.202 harbor.test.com gitlab.test.com minio.test.com jenkins.test.com
192.168.1.203 harbor.test.com gitlab.test.com minio.test.com jenkins.test.com

[admin@k8s-master-01 harbor]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.201 k8s-master-01
192.168.1.202 k8s-worker-01 jenkins.test.com harbor.test.com gitlab.test.com minio.test.com
192.168.1.203 k8s-worker-02

[admin@k8s-worker-01 ~]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.201 k8s-master-01 
192.168.1.202 k8s-worker-01 jenkins.test.com harbor.test.com gitlab.test.com minio.test.com
192.168.1.203 k8s-worker-02

[admin@k8s-worker-02 ~]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.201 k8s-master-01 
192.168.1.202 k8s-worker-01 jenkins.test.com harbor.test.com gitlab.test.com minio.test.com
192.168.1.203 k8s-worker-02
1.2 git 客户端配置(如需)
git config --global http.sslVerify false
1.3 创建 TLS 证书
# 创建tls证书
# gitlab
[admin@k8s-master-01 gitlab]$ openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout gitlab.test.com.key -out gitlab.test.com.crt \
-subj "/CN=gitlab.test.com" \
-addext "subjectAltName=DNS:gitlab.test.com,DNS:gitlab,IP:192.168.1.202"

# minio
[admin@k8s-master-01 gitlab]$ openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout minio.test.com.key -out minio.test.com.crt \
-subj "/CN=minio.test.com" \
-addext "subjectAltName=DNS:minio.test.com,DNS:minio,IP:192.168.1.202"

# registry
[admin@k8s-master-01 gitlab]$ openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout registry.test.com.key -out registry.test.com.crt \
-subj "/CN=registry.test.com" \
-addext "subjectAltName=DNS:registry.test.com,DNS:registry,IP:192.168.1.202"

# kas
[admin@k8s-master-01 gitlab]$ openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout kas.test.com.key -out kas.test.com.crt \
-subj "/CN=kas.test.com" \
-addext "subjectAltName=DNS:kas.test.com,DNS:kas,IP:192.168.1.202"


[admin@k8s-master-01 gitlab]$ ll
总用量 32
-rw-rw-r-- 1 admin admin 1188 626 11:36 gitlab.test.com.crt
-rw------- 1 admin admin 1704 626 11:36 gitlab.test.com.key
-rw-rw-r-- 1 admin admin 1172 626 11:39 kas.test.com.crt
-rw------- 1 admin admin 1704 626 11:39 kas.test.com.key
-rw-rw-r-- 1 admin admin 1180 626 11:38 minio.test.com.crt
-rw------- 1 admin admin 1704 626 11:38 minio.test.com.key
-rw-rw-r-- 1 admin admin 1200 626 11:38 registry.test.com.crt
-rw------- 1 admin admin 1704 626 11:38 registry.test.com.key

[admin@k8s-master-01 gitlab]$ sudo cp *.crt  /etc/pki/ca-trust/source/anchors/

[admin@k8s-master-01 gitlab]$ kubectl create ns gitlab
namespace/gitlab created
[admin@k8s-master-01 gitlab]$ kubectl -n gitlab create secret tls gitlab-tls --key ./gitlab.test.com.key --cert gitlab.test.com.crt
secret/gitlab-tls created
[admin@k8s-master-01 gitlab]$ kubectl -n gitlab create secret tls registry-gitlab-tls --key ./registry.test.com.key --cert registry.test.com.crt
secret/registry-gitlab-tls created
[admin@k8s-master-01 gitlab]$ kubectl -n gitlab create secret tls minio-gitlab-tls    --key ./minio.test.com.key --cert minio.test.com.crt
secret/minio-gitlab-tls created
[admin@k8s-master-01 gitlab]$ 
[admin@k8s-master-01 gitlab]$ kubectl -n gitlab create secret tls kas-gitlab-tls    --key ./kas.test.com.key --cert kas.test.com.crt
secret/kas-gitlab-tls created
1.4 邮箱认证通知准备工作
[admin@k8s-master-01 gitlab]$ kubectl create secret generic smtp-password --from-literal=password=你的邮箱授权码 -n gitlab
secret/smtp-password created
1.5 获取安装包

点击进入artifacthub官网

[admin@k8s-master-01 gitlab]$ helm repo add gitlab http://charts.gitlab.io/
[admin@k8s-master-01 gitlab]$ helm search repo gitlab/gitlab --versions | head -n 11
NAME                    CHART VERSION   APP VERSION     DESCRIPTION                                       
gitlab/gitlab           9.1.1           v18.1.1         GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           9.1.0           v18.1.0         GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           9.0.3           v18.0.3         GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           9.0.2           v18.0.2         GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           9.0.1           v18.0.1         GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           9.0.0           v18.0.0         GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           8.11.6          v17.11.5        GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           8.11.5          v17.11.5        GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           8.11.4          v17.11.4        GitLab is the most comprehensive AI-powered Dev...
gitlab/gitlab           8.11.3          v17.11.3        GitLab is the most comprehensive AI-powered Dev...
[admin@k8s-master-01 gitlab]$ helm pull gitlab/gitlab --version 9.1.1 
[admin@k8s-master-01 gitlab]$ tar -xvf gitlab-9.1.1.tgz

2. 部署安装


# 部署安装(域名方式)
[admin@k8s-master-01 gitlab]$ helm install  gitlab ./gitlab -n gitlab  --create-namespace \
--set certmanager.install=false \
--set global.image.registry=harbor.test.com/docker-hub-proxy \
--set global.hosts.domain=test.com \
--set global.ingress.configureCertmanager=false \
--set global.ingress.tls.enabled=true \
--set global.ingress.tls.secretName=gitlab-tls \
--set global.minio.enabled=true \
--set global.minio.credentials.secret=minio-gitlab-tls \
--set global.kas.enabled=true \
--set global.kas.tls.enabled=true \
--set global.kas.tls.secretName=kas-gitlab-tls \
--set global.registry.tls.enabled=true \
--set global.registry.tls.secretName=registry-gitlab-tls \
--set global.smtp.enabled=true \
--set global.smtp.address="smtp.qq.com" \
--set global.smtp.port=465 \
--set global.smtp.user_name="66666666@qq.com" \
--set global.smtp.password.secret="smtp-password" \
--set global.smtp.password.key=password \
--set global.smtp.domain="qq.com" \
--set global.smtp.authentication="login" \
--set global.smtp.openssl_verify_mode="peer" \
--set global.smtp.pool=true \
--set global.email.from="66666666@qq.com" \
--set global.email.display_name=GitLab \
--set global.email.reply_to="66666666@qq.com" \
--set certmanager-issuer.email="66666666@qq.com" \
--set prometheus.install=false

I0702 09:45:23.243263   44038 warnings.go:110] "Warning: volume \"registry-secrets\" (Projected): overlapping paths: \"httpSecret\" (Secret \"gitlab-registry-httpsecret\") with \"httpSecret\" (Secret \"gitlab-registry-httpsecret\")"
NAME: gitlab
LAST DEPLOYED: Wed Jul  2 09:44:57 2025
NAMESPACE: gitlab
STATUS: deployed
REVISION: 1
NOTES:
=== CRITICAL
The following charts are included for evaluation purposes only. They will not be supported by GitLab Support
for production workloads. Use Cloud Native Hybrid deployments for production. For more information visit
https://docs.gitlab.com/charts/installation/index.html#use-the-reference-architectures.
- PostgreSQL
- Redis
- Gitaly
- MinIO

=== NOTICE
The minimum required version of PostgreSQL is now 14. See https://docs.gitlab.com/charts/installation/upgrade.html for more details.

=== NOTICE
You've installed GitLab Runner without the ability to use 'docker in docker'.
The GitLab Runner chart (gitlab/gitlab-runner) is deployed without the `privileged` flag by default for security purposes. This can be changed by setting `gitlab-runner.runners.privileged` to `true`. Before doing so, please read the GitLab Runner chart's documentation on why we
chose not to enable this by default. See https://docs.gitlab.com/runner/install/kubernetes.html#running-docker-in-docker-containers-with-gitlab-runners
Help us improve the installation experience, let us know how we did with a 1 minute survey:https://gitlab.fra1.qualtrics.com/jfe/form/SV_6kVqZANThUQ1bZb?installation=helm&release=18-1

=== WARNING
certmanager:
    The configuration key `certmanager.install` has been renamed to `installCertmanager`.
    Please update your values. The old value will be removed in GitLab 18.3/chart 9.3
    to update the bundled certmanager and to enable schema validation.

# 查看部署详情
[admin@k8s-master-01 ~]$ kubectl get all -n gitlab
NAME                                                   READY   STATUS      RESTARTS      AGE
pod/gitlab-gitaly-0                                    1/1     Running     0             118s
pod/gitlab-gitlab-exporter-f5cbbb4fb-xwff4             1/1     Running     0             118s
pod/gitlab-gitlab-runner-755648bb58-4l5zp              1/1     Running     0             118s
pod/gitlab-gitlab-shell-69bc59d659-5tspr               1/1     Running     0             118s
pod/gitlab-gitlab-shell-69bc59d659-8fqwr               1/1     Running     0             103s
pod/gitlab-kas-6446c797f5-nwrvk                        1/1     Running     3 (95s ago)   118s
pod/gitlab-kas-6446c797f5-tchsz                        1/1     Running     2 (97s ago)   103s
pod/gitlab-migrations-75b647f-n6wdr                    0/1     Completed   1             118s
pod/gitlab-minio-6f68559cdf-k5h6f                      1/1     Running     0             118s
pod/gitlab-minio-create-buckets-628cb63-7wxdk          0/1     Completed   0             118s
pod/gitlab-nginx-ingress-controller-558557775c-5rtj4   1/1     Running     0             118s
pod/gitlab-nginx-ingress-controller-558557775c-jcsl7   1/1     Running     0             118s
pod/gitlab-postgresql-0                                2/2     Running     0             118s
pod/gitlab-redis-master-0                              2/2     Running     0             118s
pod/gitlab-registry-5ff5bc884f-pjl5d                   1/1     Running     0             103s
pod/gitlab-registry-5ff5bc884f-pz2pr                   1/1     Running     0             118s
pod/gitlab-sidekiq-all-in-1-v2-56d66d94bd-2lh2q        1/1     Running     0             118s
pod/gitlab-toolbox-7bc96fd9d-wmh5k                     1/1     Running     0             118s
pod/gitlab-webservice-default-5bddcbbb54-mdgd9         2/2     Running     0             103s
pod/gitlab-webservice-default-5bddcbbb54-p76cg         2/2     Running     0             118s

NAME                                              TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                   AGE
service/gitlab-gitaly                             ClusterIP      None             <none>        8075/TCP,9236/TCP                         118s
service/gitlab-gitlab-exporter                    ClusterIP      10.96.166.15     <none>        9168/TCP                                  118s
service/gitlab-gitlab-shell                       ClusterIP      10.100.46.60     <none>        22/TCP                                    118s
service/gitlab-kas                                ClusterIP      10.109.28.18     <none>        8150/TCP,8153/TCP,8154/TCP,8151/TCP       118s
service/gitlab-minio-svc                          ClusterIP      10.96.169.6      <none>        9000/TCP                                  118s
service/gitlab-nginx-ingress-controller           LoadBalancer   10.109.162.118   <pending>     80:30381/TCP,443:32308/TCP,22:31564/TCP   118s
service/gitlab-nginx-ingress-controller-metrics   ClusterIP      10.97.29.182     <none>        10254/TCP                                 118s
service/gitlab-postgresql                         ClusterIP      10.104.140.219   <none>        5432/TCP                                  118s
service/gitlab-postgresql-hl                      ClusterIP      None             <none>        5432/TCP                                  118s
service/gitlab-postgresql-metrics                 ClusterIP      10.106.211.210   <none>        9187/TCP                                  118s
service/gitlab-redis-headless                     ClusterIP      None             <none>        6379/TCP                                  118s
service/gitlab-redis-master                       ClusterIP      10.98.184.70     <none>        6379/TCP                                  118s
service/gitlab-redis-metrics                      ClusterIP      10.108.75.173    <none>        9121/TCP                                  118s
service/gitlab-registry                           ClusterIP      10.106.17.160    <none>        5000/TCP                                  118s
service/gitlab-webservice-default                 ClusterIP      10.108.250.92    <none>        8080/TCP,8181/TCP,8083/TCP                118s

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/gitlab-gitlab-exporter            1/1     1            1           118s
deployment.apps/gitlab-gitlab-runner              1/1     1            1           118s
deployment.apps/gitlab-gitlab-shell               2/2     2            2           118s
deployment.apps/gitlab-kas                        2/2     2            2           118s
deployment.apps/gitlab-minio                      1/1     1            1           118s
deployment.apps/gitlab-nginx-ingress-controller   2/2     2            2           118s
deployment.apps/gitlab-registry                   2/2     2            2           118s
deployment.apps/gitlab-sidekiq-all-in-1-v2        1/1     1            1           118s
deployment.apps/gitlab-toolbox                    1/1     1            1           118s
deployment.apps/gitlab-webservice-default         2/2     2            2           118s

NAME                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/gitlab-gitlab-exporter-f5cbbb4fb             1         1         1       118s
replicaset.apps/gitlab-gitlab-runner-755648bb58              1         1         1       118s
replicaset.apps/gitlab-gitlab-shell-69bc59d659               2         2         2       118s
replicaset.apps/gitlab-kas-6446c797f5                        2         2         2       118s
replicaset.apps/gitlab-minio-6f68559cdf                      1         1         1       118s
replicaset.apps/gitlab-nginx-ingress-controller-558557775c   2         2         2       118s
replicaset.apps/gitlab-registry-5ff5bc884f                   2         2         2       118s
replicaset.apps/gitlab-sidekiq-all-in-1-v2-56d66d94bd        1         1         1       118s
replicaset.apps/gitlab-toolbox-7bc96fd9d                     1         1         1       118s
replicaset.apps/gitlab-webservice-default-5bddcbbb54         2         2         2       118s

NAME                                   READY   AGE
statefulset.apps/gitlab-gitaly         1/1     118s
statefulset.apps/gitlab-postgresql     1/1     118s
statefulset.apps/gitlab-redis-master   1/1     118s

NAME                                                             REFERENCE                               TARGETS               MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/gitlab-gitlab-shell          Deployment/gitlab-gitlab-shell          cpu: <unknown>/100m   2         10        2          118s
horizontalpodautoscaler.autoscaling/gitlab-kas                   Deployment/gitlab-kas                   cpu: <unknown>/100m   2         10        2          118s
horizontalpodautoscaler.autoscaling/gitlab-registry              Deployment/gitlab-registry              cpu: <unknown>/75%    2         10        2          118s
horizontalpodautoscaler.autoscaling/gitlab-sidekiq-all-in-1-v2   Deployment/gitlab-sidekiq-all-in-1-v2   cpu: <unknown>/350m   1         10        1          118s
horizontalpodautoscaler.autoscaling/gitlab-webservice-default    Deployment/gitlab-webservice-default    cpu: <unknown>/1      2         10        2          118s

NAME                                            STATUS     COMPLETIONS   DURATION   AGE
job.batch/gitlab-migrations-75b647f             Complete   1/1           92s        118s
job.batch/gitlab-minio-create-buckets-628cb63   Complete   1/1           10s        118s



[admin@k8s-master-01 ~]$ kubectl get ingress -n gitlab 
NAME                        CLASS          HOSTS               ADDRESS         PORTS     AGE
gitlab-kas                  gitlab-nginx   kas.test.com        10.109.140.92   80, 443   2m29s
gitlab-minio                gitlab-nginx   minio.test.com      10.109.140.92   80, 443   2m29s
gitlab-registry             gitlab-nginx   registry.test.com   10.109.140.92   80, 443   2m29s
gitlab-webservice-default   gitlab-nginx   gitlab.test.com     10.109.140.92   80, 443   2m29s

3. 网络相关配置

3.1 暴露端口增加externalIPs
# 将type: LoadBalancer 改成 type: NodePort,保存退出后,会看到gitlab-shell
[admin@k8s-master-01 software]$ kubectl edit svc gitlab-nginx-ingress-controller -n gitlab 
spec:
  externalIPs:
  - 192.168.100.2
  # 根据自己需要配置
  type: NodePort

[admin@k8s-master-01 ~]$ kubectl get svc,ingress -n gitlab
NAME                                              TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                                   AGE
service/gitlab-gitaly                             ClusterIP      None             <none>          8075/TCP,9236/TCP                         3m40s
service/gitlab-gitlab-exporter                    ClusterIP      10.96.166.15     <none>          9168/TCP                                  3m40s
service/gitlab-gitlab-shell                       ClusterIP      10.100.46.60     <none>          22/TCP                                    3m40s
service/gitlab-kas                                ClusterIP      10.109.28.18     <none>          8150/TCP,8153/TCP,8154/TCP,8151/TCP       3m40s
service/gitlab-minio-svc                          ClusterIP      10.96.169.6      <none>          9000/TCP                                  3m40s
service/gitlab-nginx-ingress-controller           LoadBalancer   10.109.162.118   192.168.100.2   80:30381/TCP,443:32308/TCP,22:31564/TCP   3m40s
service/gitlab-nginx-ingress-controller-metrics   ClusterIP      10.97.29.182     <none>          10254/TCP                                 3m40s
service/gitlab-postgresql                         ClusterIP      10.104.140.219   <none>          5432/TCP                                  3m40s
service/gitlab-postgresql-hl                      ClusterIP      None             <none>          5432/TCP                                  3m40s
service/gitlab-postgresql-metrics                 ClusterIP      10.106.211.210   <none>          9187/TCP                                  3m40s
service/gitlab-redis-headless                     ClusterIP      None             <none>          6379/TCP                                  3m40s
service/gitlab-redis-master                       ClusterIP      10.98.184.70     <none>          6379/TCP                                  3m40s
service/gitlab-redis-metrics                      ClusterIP      10.108.75.173    <none>          9121/TCP                                  3m40s
service/gitlab-registry                           ClusterIP      10.106.17.160    <none>          5000/TCP                                  3m40s
service/gitlab-webservice-default                 ClusterIP      10.108.250.92    <none>          8080/TCP,8181/TCP,8083/TCP                3m40s

NAME                                                  CLASS          HOSTS               ADDRESS         PORTS     AGE
ingress.networking.k8s.io/gitlab-kas                  gitlab-nginx   kas.test.com        192.168.100.2   80, 443   3m40s
ingress.networking.k8s.io/gitlab-minio                gitlab-nginx   minio.test.com      192.168.100.2   80, 443   3m40s
ingress.networking.k8s.io/gitlab-registry             gitlab-nginx   registry.test.com   192.168.100.2   80, 443   3m40s
ingress.networking.k8s.io/gitlab-webservice-default   gitlab-nginx   gitlab.test.com     192.168.100.2   80, 443   3m40s
3.2 配置集群内部 dns 域名解析
[admin@k8s-master-01 gitlab]$ kubectl edit configmap coredns  -n kube-system
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
           lameduck 5s
        }
        ready
        hosts {
           192.168.100.1  harbor.test.com
           192.168.100.2  gitlab.test.com minio.test.com
           192.168.100.3  jenkins.test.com
           192.168.100.4  mysql.test.com
           192.168.100.5  redis.test.com
           192.168.100.6  elasticsearch.test.com
           fallthrough
        }
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }

        prometheus :9153
        forward . /etc/resolv.conf {
           max_concurrent 1000
        }
        cache 30 {
           disable success cluster.local
           disable denial cluster.local
        }
        loop
        reload
        loadbalance
    }
kind: ConfigMap
metadata:
  creationTimestamp: "2025-03-12T14:08:21Z"
  name: coredns
  namespace: kube-system
  resourceVersion: "264"
  uid: 22bfe350-733d-4d51-ae10-298be5ad9730

# 查看当前coredns的pod
[admin@k8s-master-01 gitlab]$ kubectl get pods -n kube-system
NAME                                    READY   STATUS    RESTARTS       AGE
coredns-6766b7b6bb-lxhwq                1/1     Running   0              3m45s
coredns-6766b7b6bb-tdprz                1/1     Running   0              4m12s
etcd-k8s-master-01                      1/1     Running   35 (16h ago)   96d
kube-apiserver-k8s-master-01            1/1     Running   16 (16h ago)   96d
kube-controller-manager-k8s-master-01   1/1     Running   36 (16h ago)   96d
kube-proxy-887zj                        1/1     Running   34 (16h ago)   96d
kube-proxy-kgftq                        1/1     Running   26 (38h ago)   96d
kube-proxy-rzpcw                        1/1     Running   26 (38h ago)   96d
kube-scheduler-k8s-master-01            1/1     Running   36 (16h ago)   96d

# 删除旧的coredns,删玩会自动重建coreDns
[admin@k8s-master-01 gitlab]$ kubectl delete pod -n kube-system -l k8s-app=kube-dns
pod "coredns-6766b7b6bb-lxhwq" deleted
pod "coredns-6766b7b6bb-tdprz" deleted

# 测试
kubectl run netshoot --rm -it --image=nicolaka/netshoot -- bash

4. 获取初始化密码

[admin@k8s-master-01 gitlab]$ kubectl get secrets -n gitlab
NAME                                  TYPE                 DATA   AGE
gitlab-gitaly-secret                  Opaque               1      137m
gitlab-gitlab-initial-root-password   Opaque               1      137m
gitlab-gitlab-kas-secret              Opaque               1      137m
gitlab-gitlab-runner-secret           Opaque               2      137m
gitlab-gitlab-shell-host-keys         Opaque               6      137m
gitlab-gitlab-shell-secret            Opaque               1      137m
gitlab-gitlab-suggested-reviewers     Opaque               1      137m
gitlab-gitlab-workhorse-secret        Opaque               1      137m
gitlab-kas-private-api                Opaque               1      137m
gitlab-kas-websocket-token            Opaque               1      137m
gitlab-postgresql-password            Opaque               2      137m
gitlab-rails-secret                   Opaque               1      137m
gitlab-redis-secret                   Opaque               1      137m
gitlab-registry-httpsecret            Opaque               1      137m
gitlab-registry-notification          Opaque               1      137m
gitlab-registry-secret                Opaque               2      137m
gitlab-tls                            kubernetes.io/tls    2      142m
gitlab-zoekt-basicauth                Opaque               2      137m
kas-gitlab-tls                        kubernetes.io/tls    2      141m
minio-gitlab-tls                      kubernetes.io/tls    4      141m
registry-gitlab-tls                   kubernetes.io/tls    2      142m
sh.helm.release.v1.gitlab.v1          helm.sh/release.v1   1      28m
smtp-password                         Opaque               1      140m


# 获取gitlab root用户密码
[admin@k8s-master-01 gitlab]$ kubectl get secret gitlab-gitlab-initial-root-password -n gitlab -o jsonpath='{.data.password}' | base64 -d
AaHX8vQ9y01f9VY11T9htqNKKAqRiljWVdHfEq9DIYdWBtJM4EmuVZ2c3SZHVyGF

# 获取minio 初始化key 和 密码
[admin@k8s-master-01 gitlab]$ kubectl get secrets minio-gitlab-tls -n gitlab -o yaml
apiVersion: v1
data:
  accesskey: dFp4QXdJbEdhRFhkNTR5amtJQ296UXZNN09va1RiRERqS25QQ2ZmV2JwbldScjZndzdXek1DQk5MVVIzYThSMg==
  secretkey: YThjZFJzcjMzSG9INGlKSll1SGdmdmZOMFBEZmZBd3o5WDN1Zmp0aFI5anZuQlV1bk5wYTRPVjNoMXdrdlJuSQ==
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURPekNDQWlPZ0F3SUJBZ0lVV3YvYjIxNlMycWQ4WGRtVFBwSkhFK3ZoVmtnd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0dURVhNQlVHQTFVRUF3d09iV2x1YVc4dWRHVnpkQzVqYjIwd0hoY05NalV3TnpBeU1ERTBNREU0V2hjTgpNelV3TmpNd01ERTBNREU0V2pBWk1SY3dGUVlEVlFRRERBNXRhVzVwYnk1MFpYTjBMbU52YlRDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxvSkpnY0xkVFJnYUpPdkw1T29Pdk9QKzIxa3hmZ1IKYzJ1NkJJSTJ6MGRaMEJ2MTI4QmJCOWQrcHAzK21QWWJweHpodGEwNFJVYUh6VzFMTmt6RWRNVVdPTmh2aGltawpKSWRTNlZhaFhKb2c1K2ZRS0FCZjAyRXZucytqVFFIS2JENWs0SndhdEUwT1JoOVlJNGhSd1hmRUFTNFRnd1RBCjM1L0Z2RTllTjIvL2FvMk5hQVN4NTB3Umx3OFVLc08rVnJGN2FXTk85K3lWVFVaK0JGazNzcGtGRzN0U2NXb20KSnJXSGErRlJ6dHRickdFYVN3NHE0dWd3SzVQd1R4SUZUMDNmaTEva2dPeXdOMWd1UEZuVGp3YmlmL2tVSDRPagpFRWJ5UHY5SE03djNzSkY2WUNGWllxekJnRHNqMW9IU0ZLZ3FST0xXQ3N6ZUkwb20zWEpTclkwQ0F3RUFBYU43Ck1Ia3dIUVlEVlIwT0JCWUVGSVdQMGl3ZXRZb2ZWaG93Y0I0dzZ0bDhvamY4TUI4R0ExVWRJd1FZTUJhQUZJV1AKMGl3ZXRZb2ZWaG93Y0I0dzZ0bDhvamY4TUE4R0ExVWRFd0VCL3dRRk1BTUJBZjh3SmdZRFZSMFJCQjh3SFlJTwpiV2x1YVc4dWRHVnpkQzVqYjIyQ0JXMXBibWx2aHdUQXFBSEtNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJJCnNKUUtadlRYWm9XS2o3OHJGR2lZY2MrRzVoNlkvRXpmNkdqZkdzc3VuaTJveHAvbXV6R2RDMEo1YTlydElYeTIKcXhYM1ZLVXVUU3RaT3kyNzNIMHpka3p2OW1PWDVXd21GTjJMZjJ6QW1VMFp0d005bkZVeUZwdzhhZEorVDBrdwpSQk9PTVBBQmxSb1c1WnVyVGdnT294RWZmZWlER2J1K1d6Vkh0Wkl3Ynhrb2NLQ1AzcHBNUC90T09RY0xvZzNYClpxTkZ5aHhoODZYMVJMcXlaVXBzQVltQURoQXUyUW9LNHlEQWhwYmYwV2YyOGlaRkJPdENXQ0tjLzErd05XMWkKQ0FIZjFhd3BxaVdXRDFDRGlyYWxUdHFyTmNkd2hCUTNwZVhWRnJsOERVcW9wVUN4YzgxNHU4bVMrczBEMzhxSApSc01jTFBGUkdDYUVVM0VJUHYrdwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2Z0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktnd2dnU2tBZ0VBQW9JQkFRQzZDU1lIQzNVMFlHaVQKcnkrVHFEcnpqL3R0Wk1YNEVYTnJ1Z1NDTnM5SFdkQWI5ZHZBV3dmWGZxYWQvcGoyRzZjYzRiV3RPRVZHaDgxdApTelpNeEhURkZqalliNFlwcENTSFV1bFdvVnlhSU9mbjBDZ0FYOU5oTDU3UG8wMEJ5bXcrWk9DY0dyUk5Ea1lmCldDT0lVY0YzeEFFdUU0TUV3TitmeGJ4UFhqZHYvMnFOaldnRXNlZE1FWmNQRkNyRHZsYXhlMmxqVHZmc2xVMUcKZmdSWk43S1pCUnQ3VW5GcUppYTFoMnZoVWM3Ylc2eGhHa3NPS3VMb01DdVQ4RThTQlU5TjM0dGY1SURzc0RkWQpManhaMDQ4RzRuLzVGQitEb3hCRzhqNy9Sek83OTdDUmVtQWhXV0tzd1lBN0k5YUIwaFNvS2tUaTFnck0zaU5LCkp0MXlVcTJOQWdNQkFBRUNnZ0VBQVlDWjZVUW5LK2RjYU5icEZqTHRXUEt3ZDl0d3V0WmZQWDg0MHBsaE5xVUwKeUhJRmxvblZxQ0NhZFZ1dFBnMk9uYTNYOHdCcGxNNTdjdmJBeDBGdDVQcVBOblkrR2Q5OHVRZ0xJTnM0UFhsbgpEZ2p4RGNHNFdnaFJxc0VabWRwWjRlNGNIejZhOXFWWEhTRXYxa1c4djN3YWUxWUdnSmNUSnozb1RhYkg0VlJsCjY4aWFwVk9KblUyL1lTZjllSVBoQTNCZ1gvVXptWWNMV2NFVTJjajJYQ0hkczl6ZFlkaHR1ckNKM3JxdHFuclEKNzAwL3Z5YVRGUVVXQlkwRkRRbkFYMVQ0NzV1QlkyRzF6WERJZW1CdkQvTDNKdmdNY09QT2FqVlg1ODk3eXF1VgpXZmF6Q3VKM3gzaHZ5Y3ZEcjhkc2xESkJxcGh3dzdDelFGdGMreFl1c1FLQmdRRHhMNWZESytUTVRsSE9Bd21ZClNXM1ppcCtjNG1KV0VtTmJCWWFVR1h2aHdJcHd2NzhsSUtXMktoQTh5SStCQXFnOVhUY1ozMmhyK2ttUUFsbWkKcVZNa0d6SC9FSEwxSm5ENWdXRytiQjZJZ29NUFpBT050MWlVb1VJck80czBwNUFid1NPMnJ4eklMMDl4UFhkWApuYmE1bjBQcHFOY1F1b09PSC96RGNaRmh2UUtCZ1FERmRsL1oySmR4YUxiZWYxaS84WC9CemdnRzNyblF5QnVEClYxN21kVVgyeFVONzB1R3BEblhpQ1hZd0RqY01XVi9xVFhGQnJWV2p2ME5XZEM3OXBLWmNmdEM2bGhlelFEYXcKQ29OOStOLzd6N0k1bDcrMnRjelFoN0czRDV2SDNQWEhsSFNtQlFraGVSdlZqejQ0UDlZRFRpWW1reEVRQkZmegpWcHdYbTlMd0VRS0JnUURhaU5za0Z3T0xNM1BnRE5TR3poSGd5VUJ5c0pXVms3ZSs0N1RiUUhRRlM2YWxIcmlkCmRZVnhGNC82NDJkVDZBcDAvNTlMckNnN25sOUZRenZnclRnT1FTMytCUE9FdWI0bjlWbVJadm9MaXVZaDdlZGkKWXp5NS81dXEvSnJYclBSZ2taSkFsMDZtL0djcEhoNmludDdRTmtVYzFNVjUvT1h5dzMwTndqdU9xUUtCZ1FDYwozeEw4L1U0ckFkYXQ0cHUrT1h5V21qalVSR040azFFUVFmd1p1RGN4K2J0SUJ0dlJzN1JCTStVWi9tRUxSWUdtCm9ZbUZkTkFoZTVSaWZZRExidC9mZ2J3UFQ2aGFkbnc3MU01cjZLa0E0eHNqR0FSdGdmYkovOU5XckdiRkhDeUMKTnRXSkExMGlvNVZTVHk5Ti9xRG1GVkxIdUhTSXRmcXRUalJKcVRpbUVRS0JnRnNUMXdVNGU2bXNlRWIxVlZLNgpVRjdIaVdRajU2Y2tjdVFKNTZiZGpDUjdOUHhhdkFxc3BiRWNaU3RlZkpJVXBtVGsxcVU1amwxdDBIMlVkeDB5Ck9tckNEUkpuOFMxOFAyejZaN0xBTXlpME8ycWFab1NaS0cxT1NrTFNhVUtSSzlSOGU1Tm1XbVpVNWhXc0RJTkoKUmNLbkJndjdiS2lqSUVVczhTQy9RV0tPCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K
kind: Secret
metadata:
  creationTimestamp: "2025-07-02T01:41:10Z"
  labels:
    app: gitlab
    chart: gitlab-9.1.1
    heritage: Helm
    release: gitlab
  name: minio-gitlab-tls
  namespace: gitlab
  resourceVersion: "132442"
  uid: 3991a94f-27e3-4021-a07e-f3f8845f2c9e
type: kubernetes.io/tls


[admin@k8s-master-01 gitlab]$ echo 'dFp4QXdJbEdhRFhkNTR5amtJQ296UXZNN09va1RiRERqS25QQ2ZmV2JwbldScjZndzdXek1DQk5MVVIzYThSMg==' | base64 -d
tZxAwIlGaDXd54yjkICozQvM7OokTbDDjKnPCffWbpnWRr6gw7WzMCBNLUR3a8R2
[admin@k8s-master-01 gitlab]$ 
[admin@k8s-master-01 gitlab]$ echo 'YThjZFJzcjMzSG9INGlKSll1SGdmdmZOMFBEZmZBd3o5WDN1Zmp0aFI5anZuQlV1bk5wYTRPVjNoMXdrdlJuSQ==' | base64 -d
a8cdRsr33HoH4iJJYuHgfvfN0PDffAwz9X3ufjthR9jvnBUunNpa4OV3h1wkvRnI
[admin@k8s-master-01 gitlab]$ 

5. 访问测试

5.1 页面访问登录
  • gitlab

  • minio 登录

5.2 客户端与 gitlab 建立连接测试
# 步骤1:客户机生成ssh秘钥
C:\Users\changmeidong>ssh-keygen -t rsa -C "window@test.com"
Generating public/private rsa key pair.
Enter file in which to save the key (C:\Users\changmeidong/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in C:\Users\changmeidong/.ssh/id_rsa
Your public key has been saved in C:\Users\changmeidong/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:l2YvquVyQ+op49NNWJalb4LOBZjV6hK+bzc2h9Smcto window@test.com
The key's randomart image is:
+---[RSA 3072]----+
|       .         |
|      . . .      |
|     + . +       |
|    + o =  .     |
|   . o *So=      |
|    o + *+=.     |
|     * B.*. .    |
|    = O=%...     |
|   ..B*XE=       |
+----[SHA256]-----+

# 步骤2:将生成的公钥(id_rsa.pub)粘贴到gitlab的ssh key中配置好
# 步骤3:连接测试
C:\Users\changmeidong>ssh -T git@gitlab.test.com -p 31564
The authenticity of host '[gitlab.test.com]:31564 ([192.168.1.202]:31564)' can't be established.
ED25519 key fingerprint is SHA256:hTp2MmGtTJ+4mSaYTezgvHBkEaszuKBfhXyPVqnJSvA.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[gitlab.test.com]:31564' (ED25519) to the list of known hosts.
Welcome to GitLab, @root!
5.3 集群内部与 gitlab 建立连接测试
[admin@k8s-master-01 gitlab]$ kubectl run netshoot --rm -it --image=nicolaka/netshoot -- bash
If you don't see a command prompt, try pressing enter.
netshoot:~# ssh-keygen -t rsa -C "netshoot@test.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase for "/root/.ssh/id_rsa" (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:CLUvbX1CvOmzClNgtTLTvrEfwOJdbmhUGuJ0w5VaLfs netshoot@test.com
The key's randomart image is:
+---[RSA 3072]----+
|      . . .o     |
|     . = ++ .    |
|    . X Bo+o     |
|     = &.B.o     |
|      = S *..    |
|     . B X oE    |
|      + * *      |
|       + o +     |
|        ..o      |
+----[SHA256]-----+
# 步骤2:查看秘钥,并将生成的公钥(id_rsa.pub)粘贴到gitlab的ssh key中配置好
netshoot:~# cat ~/.ssh/id_rsa.pub 
ssh-rsa AAAAB3Nz...省略....Pcq40uLctUHE= netshoot@test.com

# 步骤3:连接测试
netshoot:~# ssh -T git@gitlab.test.com
The authenticity of host 'gitlab.test.com (192.168.100.2)' can't be established.
ED25519 key fingerprint is SHA256:hTp2MmGtTJ+4mSaYTezgvHBkEaszuKBfhXyPVqnJSvA.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'gitlab.test.com' (ED25519) to the list of known hosts.
Welcome to GitLab, @root!

6. 出现的问题

6.1 无法通过 git@地址 进行 clone,一直提示输入密码,
# 问题1:通过页面提供的地址,无法clone,一直提示输入密码,这是因为clone默认走的是ssh,也就是22端口,无法直接访问到容器内
[admin@k8s-master-01 .ssh]$ git clone git@gitlab.test.com:root/spring-boot-project.git
正克隆到 'spring-boot-project'...
The authenticity of host 'gitlab.test.com (192.168.100.2)' can't be established.
ECDSA key fingerprint is SHA256:sU3lWjJTu6FBcbkBuGjPc7+EktOltcub9uFLDBMWOAU.
ECDSA key fingerprint is MD5:f8:2e:ab:76:53:46:02:d9:f3:3b:2a:3b:94:e9:30:92.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'gitlab.test.com,192.168.100.2' (ECDSA) to the list of known hosts.
git@gitlab.test.com's password: 
Permission denied, please try again.
git@gitlab.test.com's password: 
Permission denied, please try again.
git@gitlab.test.com's password: 
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights

# 问题1 解决方式-使用 SSH 配置文件指定ssh端口,当然,如果你按照我的配置暴露端口增加externalIPs,则不会出现这个问题
[admin@k8s-master-01 .ssh]$ vim ~/.ssh/config
Host gitlab.test.com
    HostName gitlab.test.com
    Port 31564  # 你的 NodePort
    User git
[admin@k8s-master-01 .ssh]$ chmod 700 ~/.ssh/config
[admin@k8s-master-01 .ssh]$ git clone git@gitlab.test.com:root/spring-boot-project.git
正克隆到 'spring-boot-project'...
remote: Enumerating objects: 28, done.
remote: Total 28 (delta 0), reused 0 (delta 0), pack-reused 28 (from 1)
接收对象中: 100% (28/28), 13.91 KiB | 0 bytes/s, done.
处理 delta 中: 100% (5/5), done.
6.2 无法通过 https 进行 clone
[admin@k8s-master-01 .ssh]$ git clone https://gitlab.test.com/root/spring-boot-project.git
正克隆到 'spring-boot-project'...
fatal: unable to access 'https://gitlab.test.com/root/spring-boot-project.git/': Issuer certificate is invalid.

# 解决方案1:临时措施,关闭ssl认证
[admin@k8s-master-01 .ssh]$ git config --global http.sslVerify false
[admin@k8s-master-01 .ssh]$ git clone https://gitlab.test.com/root/spring-boot-project.git
正克隆到 'spring-boot-project'...
Username for 'https://gitlab.test.com': root
Password for 'https://root@gitlab.test.com': 
remote: Enumerating objects: 28, done.
remote: Total 28 (delta 0), reused 0 (delta 0), pack-reused 28 (from 1)
Unpacking objects: 100% (28/28), done.

# 解决方案2:
# 步骤1:只针对特定域名禁用验证(更安全)
[admin@k8s-master-01 .ssh]$ git config --global http.https://gitlab.test.com.sslVerify false
# 步骤2:获取并信任证书(生产环境必备)
[admin@k8s-master-01 .ssh]$ echo -n | openssl s_client -connect gitlab.test.com:443 \
  -servername gitlab.test.com 2>/dev/null \
  | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > gitlab.test.com.crt
# 让 Git 显式信任该证书
[admin@k8s-master-01 .ssh]$ git config --global http.https://gitlab.test.com.sslCAInfo /path/to/gitlab.test.com.crt
# 验证配置
[admin@k8s-master-01 .ssh]$ git config --global --get http.https://gitlab.test.com.sslCAInfo
/path/to/gitlab.test.com.crt
### Kubernetes 上部署 GitLabGitLab Runner 的最佳实践 在 Kubernetes 集群上部署 GitLabGitLab Runner 是一种高效的方式来管理持续集成/持续交付(CI/CD)流程。以下是关于此主题的一些关键点: #### 1. 使用 Helm Chart 安装 GitLab Helm 是 Kubernetes 的包管理工具,可以简化复杂应用的安装过程。官方提供了用于部署 GitLabHelm Chart,这使得配置和扩展变得简单。 通过以下命令可以从官方仓库安装 GitLab: ```bash helm repo add gitlab https://charts.gitlab.io/ helm install my-gitlab gitlab/gitlab --namespace gitlab-system ``` 上述操作会自动设置必要的组件,如 PostgreSQL 数据库、Redis 缓存服务以及 MinIO 对象存储等[^4]。 #### 2. 自定义配置文件调整资源分配 为了优化性能并满足特定需求,在实际生产环境中通常需要修改默认参数。可以通过创建自定义 values.yaml 文件来覆盖标准选项。例如增加 CPU 或内存限额给 runner pod。 示例片段如下所示: ```yaml global: runners: cpuLimit: "1" memoryLimit: "2Gi" postgresql: resources: requests: cpu: "500m" memory: "1Gi" redis: master: resources: limits: cpu: "300m" memory: "512Mi" ``` #### 3. 设置专用节点供 Runners 使用 考虑到安全性与隔离度,建议指定某些工作负载仅能在标记过的节点执行。利用 taints & tolerations 功能实现这一点非常方便快捷。 先污染目标服务器组: ```bash kubectl taint nodes <node-name> dedicated=gitlab-runner:NoSchedule ``` 接着更新 helm chart 中的相关字段允许这些 pods 调度到受保护实例之上。 #### 4. 整合外部 CI 工具链路 如果已经存在其他依赖的服务比如 SonarQube 扫描器或者 Artifactory 存储库,则应该考虑将其接入进来形成完整的流水线解决方案。这样不仅可以提升开发效率还能保障软件质量[^5]。 最后提醒一点就是记得定期备份整个系统以防万一发生意外情况造成数据丢失! ```python import subprocess def backup_gitlab(): try: result = subprocess.run(['kubectl', 'exec', '-it', '<pod_name>', '--', '/bin/bash'], check=True) print(result.stdout.decode()) except Exception as e: print(f"Error occurred during backing up:{e}") backup_gitlab() ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

哒卟溜......

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值