使用 etcdadm 部署 etcd 集群

发布时间： 更新时间： 总字数：1552 阅读时间：4m 作者：IP：上海 网址

etcdadm是一个命令行的工具，用于操作，集群。它可以很容易地创建一个新的集群，将一个成员添加或删除某成员从现有的集群。它的用户经验的启发通过 kubeadm

安装

  • 要安装 etcd 的环境参考
  • 下载 etcd
# 下载页面 https://github.com/kubernetes-sigs/etcdadm/releases
wget https://github.com/kubernetes-sigs/etcdadm/releases/download/v0.1.5/etcdadm-linux-amd64
mv etcdadm-linux-amd64 etcdadm
chmod a+x etcdadm
  • /etc/hosts 配置
172.17.0.3      etcd1
172.17.0.4      etcd2
172.17.0.5      etcd3
  • 复制 etcd 到所有节点
for ((i=1; i<=3; i++)); do
  docker cp etcdadm etcd${i}:/usr/local/bin/
done

help

etcdadm --help ...

初始化集群

etcdadm help init ...

说明：

  • etcd 证书地址 /etc/etcd/pki
  • 配置磁盘优先级 --disk-priorities
  • 默认安装目录 --install-dir /opt/bin/
  • 默认 etcd 版本 3.4.9，使用 --version 3.4.9 指定版本

在 etcd 1 操作

# 使用 https://gh-proxy.com/ 加速
$ etcdadm init --release-url https://gh-proxy.com/https://github.com/coreos/etcd/releases/download --server-cert-extra-sans "etcd1,etcd2,etcd3,172.17.0.3,172.17.0.4,172.17.0.5" --version 3.5.13
  • 详细日志如下
etcdadm init log ...
  • 当前节点信息
etcdadm info
{
  "ID": 6406391081376168596,
  "name": "etcd1",
  "peerURLs": [
    "https://172.17.0.3:2380"
  ],
  "clientURLs": [
    "https://172.17.0.3:2379"
  ]
}
  • 配置 /etc/etcd/
etcd configure ...
  • 自动生成的证书
$ ls -lhart /etc/etcd/pki
total 48K
ca.key
ca.crt
server.key
server.crt
peer.key
peer.crt
etcdctl-etcd-client.key
etcdctl-etcd-client.crt
apiserver-etcd-client.key
apiserver-etcd-client.crt
  • 查看日志
journalctl -u etcd -f
  • 启动文件
$ systemctl cat etcd.service
# /etc/systemd/system/etcd.service
[Unit]
Description=etcd
Documentation=https://github.com/coreos/etcd
Conflicts=etcd-member.service
Conflicts=etcd2.service

[Service]
EnvironmentFile=/etc/etcd/etcd.env
ExecStart=/opt/bin/etcd

Type=notify
TimeoutStartSec=0
Restart=on-failure
RestartSec=5s

LimitNOFILE=65536
Nice=-10
IOSchedulingClass=best-effort
IOSchedulingPriority=2
MemoryLow=200M

[Install]
WantedBy=multi-user.target
  • etcdctl 客户端
ln -s /opt/bin/etcdctl /usr/local/bin/

添加节点

  • 初始化目录
mkdir -p /etc/etcd/pki/ /var/cache/etcdadm/etcd/v3.5.13/
  • 从安装过的节点分发 CA 证书
rsync -avR /etc/etcd/pki/ca.* <Member IP address>:/
# rsync -avR /etc/etcd/pki/ca.* 172.17.0.4:/
  • 复制缓存的安装包，提前缓存帮助 etcdadm help download
rsync -avR /var/cache/etcdadm/etcd/v3.5.13/etcd-v3.5.13-linux-amd64.tar.gz <Member IP address>:/
# rsync -avR /var/cache/etcdadm/etcd/v3.5.13/etcd-v3.5.13-linux-amd64.tar.gz 172.17.0.4:/
  • 新节点加入集群
etcdadm help join ... 
etcdadm join <endpoint> --release-url https://gh-proxy.com/https://github.com/coreos/etcd/releases/download
etcdadm join log ...

清理节点

安装失败，或节点下线时

etcdadm reset

高级用法

从快照还原集群

etcdadm init --snapshot /path/to/etcd.snapshot

使用

$ etcdctl member list --cacert=/etc/etcd/pki/ca.crt --cert=/etc/etcd/pki/server.crt --key=/etc/etcd/pki/server.key
9d4f5cf4152016b2, started, etcd1, https://172.17.0.3:2380, https://172.17.0.3:2379, false
e541d05ce4978fd2, started, etcd2, https://172.17.0.4:2380, https://172.17.0.4:2379, false
ffde685c2fd67496, started, etcd3, https://172.17.0.5:2380, https://172.17.0.5:2379, false

F&Q

etcdadm join 失败

# 新节点日志
INFO[0000] [membership] Checking if this member was added
{"level":"warn","ts":"2024-04-21T11:06:35.761Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-40b4d603-dba4-4a01-85a8-a5127ab4bb45/172.17.0.3:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
FATA[0005] [membership] Error listing members: context deadline exceeded

# 被加入的 etcd 日志
$ journalctl -u etcd -f
...
Apr 21 11:19:50 etcd1 etcd[555]: rejected connection from "172.17.0.4:59730" (error "remote error: tls: bad certificate", ServerName "")
...
  • 查看证书信息
openssl x509 .crt ...

etcdadmin init --server-cert-extra-sans "etcd1,etcd2,etcd3,172.17.0.3,172.17.0.4,172.17.0.5" 指定集群其他节点的信息，之后可以查看到正确的证书信息

openssl x509 -in /etc/etcd/pki/server.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1973058493428135934 (0x1b61b63efdd9f7fe)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = etcd
        Validity
            Not Before: Apr 21 11:15:13 2024 GMT
            Not After : Apr 21 11:15:13 2025 GMT
        ...
        X509v3 extensions:
            ...

            X509v3 Subject Alternative Name:
                DNS:etcd1, DNS:etcd2, DNS:etcd3, DNS:etcd2, IP Address:172.17.0.3, IP Address:172.17.0.4, IP Address:172.17.0.5, IP Address:172.17.0.4, IP Address:127.0.0.1

参考

  1. https://github.com/kubernetes-sigs/etcdadm
最新评论