keepalived 做vip，k8s-新增master节点高可用_kubeadm 新增master节点-CSDN博客

安装keepalived

apt install keepalived -y

在主master节点

vim /etc/keepalived/keepalived.conf

global_defs {
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state MASTER
    nopreempt
    interface ens33
    virtual_router_id 80
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass just0kk
    }
    virtual_ipaddress {
        192.168.1.112
    }
}
#virtual_server 192.168.123.130 6443 {
#    delay_loop 6
#    lb_algo loadbalance
#    lb_kind DR
#    net_mask 255.255.255.0
#    persistence_timeout 0
#    protocol TCP
#    real_server 192.168.1.105 6443 {
#        weight 1
#        SSL_GET {
#            url {
#              path /healthz
#              status_code 200
#            }
#            connect_timeout 3
#            nb_get_retry 3
#            delay_before_retry 3
#        }
#    }
#    real_server 192.168.1.110 6443 {
#        weight 1
#        SSL_GET {
#            url {
#              path /healthz
#              status_code 200
#            }
#            connect_timeout 3
#            nb_get_retry 3
#            delay_before_retry 3
#        }
#    }
}

启动 keepalived

 systemctl enable keepalived && systemctl restart keepalived

在备用master节点，直接复制 master的 keepalived.conf，修改 state的值为 BACKUP，

global_defs {
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state BACKUP # 主节点未MASTER
    nopreempt
    interface ens33
    virtual_router_id 80
    priority 100 # 优先级
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass just0kk
    }
    virtual_ipaddress {
        192.168.1.112 # 虚拟ip的地址
    }
}
#virtual_server 192.168.123.130 6443 {
#    delay_loop 6
#    lb_algo loadbalance
#    lb_kind DR
#    net_mask 255.255.255.0
#    persistence_timeout 0
#    protocol TCP
#    real_server 192.168.1.105 6443 {
#        weight 1
#        SSL_GET {
#            url {
#              path /healthz
#              status_code 200
#            }
#            connect_timeout 3
#            nb_get_retry 3
#            delay_before_retry 3
#        }
#    }
#    real_server 192.168.1.110 6443 {
#        weight 1
#        SSL_GET {
#            url {
#              path /healthz
#              status_code 200
#            }
#            connect_timeout 3
#            nb_get_retry 3
#            delay_before_retry 3
#        }
#    }
}

在当前唯一的master节点上运行如下命令第一步：

kubeadm init phase upload-certs --upload-certs

执行结果如下：

1 # kubeadm init phase upload-certs --upload-certs
2 I1109 14:34:00.836965    5988 version.go:255] remote version is much newer: v1.25.3; falling back to: stable-1.22
3 [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
4 [upload-certs] Using certificate key:
5 ecf2abbfdf3a7bc45ddb2de75152ec12889971098d69939b98e4451b53aa3033

第二步：

kubeadm token create --print-join-command

执行结果如下

[root@k8s-master ~]# kubeadm token create --print-join-command
kubeadm join 172.16.0.1:6443 --token xxxxxx --discovery-token-ca-cert-hash sha256:xxxxxxx

第三步：将得到的token和key进行拼接，得到如下命令：

在第二部得到的结果 + --control-plane --certificate-key + 第一步得到的key

kubeadm join 172.16.0.1:6443 --token q466v0.hbk3qjreznjsf8ew --discovery-token-ca-cert-hash xxxxxxx --control-plane --certificate-key xxxxxxx

注意事项：

不要使用 --experimental-control-plane，会报错
要加上--control-plane --certificate-key ，不然就会添加为node节点而不是master
join的时候节点上不要部署，如果部署了kubeadm reset后再join

第四步： join之后在原先唯一的master节点上成功后，显示如下消息：

This node has joined the cluster and a new control plane instance was created:

* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane (master) label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
* A new etcd member was added to the local/stacked etcd cluster.

To start administering your cluster from this node, you need to run the following as a regular user:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run 'kubectl get nodes' to see this node join the cluster.

这样，我们在任何一个master节点上使用命令

kubectl get nodes

都能看到现在是全部的节点：

# kubectl get nodes
NAME          STATUS   ROLES    AGE   VERSION
k8s-master    Ready    master   57m   v1.18.0
k8s-master1   Ready    master   21m   v1.18.0
k8s-master2   Ready    master   21m   v1.18.0
k8s-node1     Ready    <none>   56m   v1.18.0
k8s-node2     Ready    <none>   56m   v1.18.0
k8s-node3     Ready    <none>   56m   v1.18.0
k8s-node4     Ready    <none>   56m   v1.18.0
k8s-node5     Ready    <none>   56m   v1.18.0

报错： 1. 第一次加入集群的时候会有以下报错：

 1 [preflight] Running pre-flight checks
 2 [preflight] Reading configuration from the cluster...
 3 [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
 4 error execution phase preflight:
 5 One or more conditions for hosting a new control plane instance is not satisfied.
 6 
 7 unable to add a new control plane instance a cluster that doesn't have a stable controlPlaneEndpoint address
 8 
 9 Please ensure that:
10 * The cluster has a stable controlPlaneEndpoint address.
11 * The certificates that must be shared among control plane instances are provided.
12 
13 
14 To see the stack trace of this error execute with --v=5 or higher

解决办法如下：

查看kubeadm-config.yaml
kubectl -n kube-system get cm kubeadm-config -oyaml
发现没有controlPlaneEndpoint
添加controlPlaneEndpoint
kubectl -n kube-system edit cm kubeadm-config
大概在这么个位置：
kind: ClusterConfiguration
kubernetesVersion: v1.18.0
controlPlaneEndpoint: 172.16.0.56:6443
然后再在准备添加为master的节点上执行kubeadm join的命令

报错2：新增的master 可能会 http:/xxx:6433 connect refuse

由于K8s必须保持全程关闭交换内存，之前我安装是只是使用swapoff -a命令暂时关闭swap。而机器重启后，swap还是会自动启用，从而导致kubelet无法启动。

把/etc/fstab 里swap 注释掉

# 不同操作系统可能不一样
sed -i 's/# \/swapfile none swap/\/swapfile none swap/g'   /etc/fstab
sed -i 's/\/swap.img.*/#\/swap.img       none    swap    sw      0       0/g' /etc/fstab

或者直接永久关闭

free # 永久关闭

sed -ri 's/.*swap.*/#&/' /etc/fstab