注：如果你的ETCD集群已经被其他集群使用过，请一定要清除原有的ETCD集群中的所有数据，否则会导致新的集群报各种莫名其妙的错误。
一、系统优化
1、转发ipv4，并让iptables看到桥接流量
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay && sudo modprobe br_netfilter
# 设置所需要的sysctl参数，并在重启后保持其参数生效
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
# 不重启使sysctl参数生效
sudo sysctl --system
#查看结果
lsmod | grep br_netfilter
lsmod | grep overlaysysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward验证net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, net.ipv4.ip_forward这三个系统变量是否已经在你的sysctl中。
2、关闭swap分区
swapoff -a && sed -i '/swap/d' /etc/fstab3、设置主机名
编辑主机文件
# vim /etc/hosts
127.0.0.1 k8smaster1
127.0.1.1 k8smaster1
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.50.31 k8smaster1.cluster.local k8smaster1
192.168.50.32 k8smaster2.cluster.local k8smaster2
192.168.50.33 k8smaster3.cluster.local k8smaster3
192.168.50.34 k8snode1.cluster.local k8snode1
192.168.50.35 k8snode2.cluster.local k8snode2
192.168.50.36 k8snode3.cluster.local k8snode3设置主机名
hostnamectl set-hostname k8smaster1.cluster.local
......二、全部节点安装kubernetes
初始三个主节点，初始化的过程是一样的。注意根据自己的主机情况修改对应的ip地址等等。
1、安装kubernetes
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
tee /etc/apt/sources.list.d/kubernetes.list <<EOF
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt update -y && \
apt -y install vim git curl wget kubelet=1.24.14-00 kubeadm=1.24.14-00 kubectl=1.24.14-00 kubernetes-cni2、安装containerd
见文档《Kubernetes—Containerd安装与配置》。
配置containerd私服
vim /etc/containerd/config.toml#更改containerd拉取镜像的私服地址
[plugins."io.containerd.grpc.v1.cri".registry.auths]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.dokbok.com/k8s"]
username = "admin"
password = "你的密码"
[plugins."io.containerd.grpc.v1.cri".registry.headers]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.dokbok.com/k8s"]
endpoint = ["https://xxxxxx.com/k8s"]
#更改pause的配置，版本号按配置文件中的来，不要做更改
sandbox_image = "harbor.dokbok.com/k8s/pause:3.6"重新启动containerd服务
systemctl restart containerd三、初始化主节点
1、安装依赖到自己的镜像仓库
查看依赖哪些镜像
kubeadm config images list这些镜像需要通过科学上网上传到自己的镜像仓库中
docker pull docker.io/calico/kube-controllers:v3.25.0
docker tag docker.io/calico/kube-controllers:v3.25.0 harbor.dokbok.com/library/kube-controllers:v3.25.0
docker push harbor.dokbok.com/library/kube-controllers:v3.25.02、配置crictl
cat > /etc/crictl.yaml <<EOF
runtime-endpoint: unix:///var/run/containerd/containerd.sock
image-endpoint: unix:///var/run/containerd/containerd.sock
timeout: 0
debug: false
pull-image-on-create: false
EOF测试命令
crictl images3、创建或转换配置文件
创建默认的初始化配置文件
kubeadm config print init-defaults如果你有原来的旧版本的配置文件可以使用下面命令进行转换
kubeadm config migrate --old-config old.yaml --new-config new.yaml4、修改配置文件
示例内容如下：
# vim k8s.yaml
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: 2bdvub.91nskybmaojzfkod
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.50.31
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
name: k8smaster1.cluster.local
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 192.168.50.31:6443
controllerManager: {}
dns: {}
etcd:
external:
endpoints:
- https://192.168.50.25:2379
- https://192.168.50.26:2379
- https://192.168.50.27:2379
caFile: /etc/etcd/cert/ca.pem
certFile: /etc/etcd/cert/kubernetes.pem
keyFile: /etc/etcd/cert/kubernetes-key.pem
imageRepository: xxxxxx.com/k8s
kind: ClusterConfiguration
kubernetesVersion: v1.24.17
networking:
dnsDomain: cluster.local
podSubnet: 10.30.0.0/12
serviceSubnet: 10.96.0.0/12
scheduler: {}5、初始化第一个主节点
kubeadm init --config k8s.yaml一定要初始化成功后系统输出的相关信息记录好，其中会包括添加主节点、工作节点的方式等，后续会用到。
6、编辑客户端
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf使用命令查看节点运行情况
kubectl get nodes
kubectl get pods -n kube-system四、将其他主节点添加到集群中
1、拷贝ETCD的证书到其他主节点
以节点二为示例，其他节点参照即可。
#在节点一上执行
scp -r /etc/etcd/ k8smaster2:/etc/2、拷贝kubernetes的pki证书到其他节点
#在节点一上执行
scp -r /etc/kubernetes/pki/ k8smaster2:~/3、删除不需要的证书内容
#在节点二上执行
rm ~/pki/apiserver.*
mv ~/pki/ /etc/kubernetes/4、执行加入节点操作
kubeadm join 192.168.50.31:6443 --token 2bdvub.91njzfkod --discovery-token-ca-cert-hash sha256:71ee10125aaa191132cd5070a4bcd709eb2fb51c7 --control-plane 注意：上面的命令会在初始第一个节点后系统输出的信息中找到。
五、添加工作节点
执行添加到集群的命令（该命令来自于创建第一个主节点时打印的信息）。
kubeadm join 192.168.50.31:6443 --token 2bdvunskybmaojzfkod \
--discovery-token-ca-cert-hash sha256:71ee10125aaa191132cdef0ffd26b1d05070a4bcd709eb2fb51c7
六、安装网络插件
七、查看集群运行状态
八、错误解决
1、no such file or directory\” runtime=io.containerd.runc.v2\n
find / -name containerd-shim-runc-v1
usr/local/bin/containerd-shim-runc-v1
cp /usr/local/bin/containerd-shim-runc-v1 /usr/local/bin/containerd-shim-runc-v22、Warning FailedScheduling 34s (x13 over 12m) default-scheduler 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn’t tolerate.
kubectl get node -o yaml |grep taint -A 5
kubectl describe node k8s1 | grep Taints
kubectl taint node k8s1 node.kubernetes.io/not-ready:NoSchedule --overwrite