1.7 准备K8S集群配置文件

vim inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml

---
# Kubernetes configuration dirs and system namespace.
# Those are where all the additional config stuff goes
# the kubernetes normally puts in /srv/kubernetes.
# This puts them in a sane location and namespace.
# Editing those values will almost surely break something.
kube_config_dir: /etc/kubernetes
kube_script_dir: "{{ bin_dir }}/kubernetes-scripts"
kube_manifest_dir: "{{ kube_config_dir }}/manifests"

# This is where all the cert scripts and certs will be located
kube_cert_dir: "{{ kube_config_dir }}/ssl"

# This is where all of the bearer tokens will be stored
kube_token_dir: "{{ kube_config_dir }}/tokens"

kube_api_anonymous_auth: true

## Change this to use another Kubernetes version, e.g. a current beta release
kube_version: v1.26.3

# Where the binaries will be downloaded.
# Note: ensure that you've enough disk space (about 1G)
local_release_dir: "/tmp/releases"
# Random shifts for retrying failed ops like pushing/downloading
retry_stagger: 5

# This is the user that owns tha cluster installation.
kube_owner: kube

修改：重点观察20、70、76、81、160行等
默认可以不用修改。

1.8 准备k8s集群插件文件

要启用 Kuberenetes 仪表板和入口控制器等插件，请在文件inventory/mycluster/group_vars/k8s_cluster/addons.yml 中将参数设置为已启用：

vim inventory/mycluster/group_vars/k8s_cluster/addons.yml
  1 ---
  2 # Kubernetes dashboard
  3 # RBAC required. see docs/getting-started.md for access details.
  4 dashboard_enabled: true
  5
  6 # Helm deployment
  7 helm_enabled: false
  8
  9 # Registry deployment
 10 registry_enabled: false
 11 # registry_namespace: kube-system
 12 # registry_storage_class: ""
 13 # registry_disk_size: "10Gi"
 14
 15 # Metrics Server deployment
 16 metrics_server_enabled: false

1.9 准备ssh密钥

# 所有节点执行
ssh-keygen
# kubespray节点执行
ssh-copy-id root@192.168.1.11
ssh-copy-id root@192.168.1.12
ssh-copy-id root@192.168.1.13
ssh-copy-id root@192.168.1.14
ssh-copy-id root@192.168.1.15

1.9.1 在K8S集群节点添加sysops用户指行授权

所有的k8s集群节点

echo "sysops ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/sysops

1.10 k8s集群主机安全设置

禁用firewalld防火墙

cd kubespray/
ansible all -i inventory/mycluster/hosts.yaml -m shell -a "systemctl stop firewalld && systemctl disable firewalld"

1.11 k8s集群主机路由转发设置

ansible all -i inventory/cluster/hosts.yaml -m shell -a "echo 'net.ipv4.ip_forward=1' | tee -a /etc/sysctl.conf"

1.12 禁用swap分区

ansible all -i inventory/mycluster/hosts.yaml -m shell -a "sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab &&  swapoff -a"

二、k8s集群部署及可用性验证

# 如果没有执行成功，可以多次执行。 这里可能会出现各种各样的问题 可以参考最后
ansible-playbook -i inventory/mycluster/hosts.yaml --become --become-user=root cluster.yml

最后

fatal: [node1]: FAILED! => {"msg": "The conditional check 'kubeadm_certificate_key is not defined' failed. The error was: An unhandled exception occurred while templating '{{ lookup('password', credentials_dir + '/kubeadm_certificate_key.creds length=64 chars=hexdigits') | lower }}'. Error was a , original message: An unhandled exception occurred while running the lookup plugin 'password'. Error was a , original message: [Errno 17] 文件已存在: b'/root/kubespray/inventory/cluster/credentials/a86e91bcbd996c1a4d7cce747921ce69f6f353d9.ansible_lockfile'. [Errno 17] 文件已存在: b'/root/kubespray/inventory/cluster/credentials/a86e91bcbd996c1a4d7cce747921ce69f6f353d9.ansible_lockfile'\n\nThe error appears to be in '/root/kubespray/roles/kubernetes/control-plane/tasks/kubeadm-setup.yml': line 210, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set kubeadm certificate key\n  ^ here\n"}
fatal: [node3]: FAILED! => {"msg": "The conditional check 'kubeadm_certificate_key is not defined' failed. The error was: An unhandled exception occurred while templating '{{ lookup('password', credentials_dir + '/kubeadm_certificate_key.creds length=64 chars=hexdigits') | lower }}'. Error was a , original message: An unhandled exception occurred while running the lookup plugin 'password'. Error was a , original message: [Errno 17] 文件已存在: b'/root/kubespray/inventory/cluster/credentials/a86e91bcbd996c1a4d7cce747921ce69f6f353d9.ansible_lockfile'. [Errno 17] 文件已存在: b'/root/kubespray/inventory/cluster/credentials/a86e91bcbd996c1a4d7cce747921ce69f6f353d9.ansible_lockfile'\n\nThe error appears to be in '/root/kubespray/roles/kubernetes/control-plane/tasks/kubeadm-setup.yml': line 210, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set kubeadm certificate key\n  ^ here\n"}

解决见：https://github.com/kubernetes-sigs/kubespray/issues/9916

修改inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml，手工指定：

kubeadm_certificate_key: "dbfEcDFfF8Cc6fcaCDfBC0c4eb6baea4FDbbee4B3fc1A252c5bfe765de6FbEDc"

如果出现证书问题无法下载文件 可以使用wget手动下载文件到/tmp下