Etcd 集群安装部署
介绍
etcd 集群的启动方式包括：
- 静态启动
- etcd 动态发现
- NDS 发现
etcd 集群架构
- Raft 层是实现 etcd 数据一致性的关键，etcd 节点之间通过 Raft 协议实现通信
- Storage 层实现持久化键值对
- WAL 日志模块
- 快照模块
图片参考
环境
- 使用 docker 启动 3 台主机，模拟 vm
for ((i=1; i<=3; i++)); do
docker run \
-it \
-d \
--privileged \
--hostname etcd${i} \
--name etcd${i} \
xiexianbin/ubuntu:20.04 \
/lib/systemd/systemd
done- IP 信息
- etcd1:
172.17.0.3 - etcd2:
172.17.0.4 - etcd3:
172.17.0.5
- etcd1:
- etcd 下载地址：https://github.com/etcd-io/etcd/releases
mkdir -p /opt/etcd ; cd /opt/etcd
wget https://github.com/etcd-io/etcd/releases/download/v3.4.27/etcd-v3.4.27-linux-amd64.tar.gz- 复制到所有节点
for ((i=1; i<=3; i++)); do
docker exec -it etcd${i} mkdir -p /opt/etcd/{data,etc}
docker cp etcd-v3.4.27-linux-amd64.tar.gz etcd${i}:/opt/etcd
docker exec -it etcd${i} bash -c "cd /opt/etcd; tar -zxf etcd-v3.4.27-linux-amd64.tar.gz; ln -s etcd-v3.4.27-linux-amd64 etcd"
done- 规划目录说明
/opt/etcd/dataetcd 保存数据的目录/opt/etcd/etc/etcd.yaml配置文件目录
配置文件
-
创建配置文件 /opt/etcd/etc/etcd.yaml，三个节点配置文件内容分别如下：
-
etcd1
name: etcd1
data-dir: /opt/etcd/data
initial-advertise-peer-urls: http://172.17.0.3:2380
listen-peer-urls: http://172.17.0.3:2380
listen-client-urls: http://172.17.0.3:2379,http://127.0.0.1:2379
advertise-client-urls: http://172.17.0.3:2379
initial-cluster-token: etcd-cluster
initial-cluster: etcd1=http://172.17.0.3:2380,etcd2=http://172.17.0.4:2380,etcd3=http://172.17.0.5:2380
initial-cluster-state: new- etcd2
name: etcd2
data-dir: /opt/etcd/data
initial-advertise-peer-urls: http://172.17.0.4:2380
listen-peer-urls: http://172.17.0.4:2380
listen-client-urls: http://172.17.0.4:2379,http://127.0.0.1:2379
advertise-client-urls: http://172.17.0.4:2379
initial-cluster-token: etcd-cluster
initial-cluster: etcd1=http://172.17.0.3:2380,etcd2=http://172.17.0.4:2380,etcd3=http://172.17.0.5:2380
initial-cluster-state: new- etcd3
name: etcd3
data-dir: /opt/etcd/data
initial-advertise-peer-urls: http://172.17.0.5:2380
listen-peer-urls: http://172.17.0.5:2380
listen-client-urls: http://172.17.0.5:2379,http://127.0.0.1:2379
advertise-client-urls: http://172.17.0.5:2379
initial-cluster-token: etcd-cluster
initial-cluster: etcd1=http://172.17.0.3:2380,etcd2=http://172.17.0.4:2380,etcd3=http://172.17.0.5:2380
initial-cluster-state: new配置参数说明：
name：etcd 节点名称data-dir：数据存储目录initial-advertise-peer-urls：集群的其他节点通过该地址与当前节点通信listen-peer-urls：当前节点通过该地址监听集群其他节点发送的信息listen-client-urls：当前节点通过该地址监听客户端发送的信息advertise-client-urls：客户端通过该地址与当前节点通信initial-cluster-token：用于区分不同的集群，同一集群的所有节点配置相同的值initial-cluster：当前集群的所有节点信息，当前节点根据此信息与其他节点取得联系initial-cluster-state：本次是否为新建集群，有两个取值：new和existing
配置 service 服务
cat >/usr/lib/systemd/system/etcd.service<<EOF
[Unit]
Description=etcd
After=network.target
[Service]
Type=notify
LimitNOFILE=40000
TimeoutStartSec=0
ExecStart=/opt/etcd/etcd/etcd --config-file=/opt/etcd/etc/etcd.yaml
[Install]
WantedBy=multi-user.target
EOF启动服务
for ((i=1; i<=3; i++)); do
docker exec -it etcd${i} systemctl daemon-reload
docker exec -it etcd${i} systemctl start etcd.service &
done验证
- 查看 etcd 版本
$ /opt/etcd/etcd/etcd --version
WARNING: Package "github.com/golang/protobuf/protoc-gen-go/generator" is deprecated.
A future release of golang/protobuf will delete this package,
which has long been excluded from the compatibility promise.
etcd Version: 3.4.27
Git SHA: c92fb80f3
Go Version: go1.19.10
Go OS/Arch: linux/amd64- etcdctl 软连接
ln -s /opt/etcd/etcd/etcdctl /usr/local/bin/etcdctl- 查看集群成员列表
$ etcdctl member list
1af57124460c9172, started, etcd3, http://172.17.0.5:2380, http://172.17.0.5:2379, false
660aa483274d103a, started, etcd1, http://172.17.0.3:2380, http://172.17.0.3:2379, false
ad0233873e2a0054, started, etcd2, http://172.17.0.4:2380, http://172.17.0.4:2379, false- 查看集群成员健康情况
$ etcdctl endpoint health --endpoints=http://172.17.0.3:2380,http://172.17.0.4:2380,http://172.17.0.5:2380
http://172.17.0.5:2380 is healthy: successfully committed proposal: took = 5.749126ms
http://172.17.0.3:2380 is healthy: successfully committed proposal: took = 5.820195ms
http://172.17.0.4:2380 is healthy: successfully committed proposal: took = 6.513411ms其他
也可将参数添加到 systemctl 中
/etc/systemd/system/etcd.service
[Unit]
Description=etcd
Documentation=https://github.com/coreos/etcd
[Service]
Type=notify
Restart=always
RestartSec=5s
LimitNOFILE=40000
TimeoutStartSec=0
EnvironmentFile=-/etc/sysconfig/etcd-ssl
ExecStart=/usr/local/bin/etcd --name etcd1 \
--data-dir /data/bkee/public/etcd \
--listen-client-urls http://<ip1>:2379,http://127.0.0.1:2379 \
--advertise-client-urls http://<ip1>:2379 \
--listen-peer-urls http://<ip1>:2380 \
--initial-advertise-peer-urls http://<ip1>:2380 \
--initial-cluster etcd=http://<ip0>:2380,etcd1=http://<ip1>:2380,etcd2=http://<ip2>:2380 \
--initial-cluster-state new \
--initial-cluster-token etcd-cluster-token \
--auto-compaction-retention 1
[Install]
WantedBy=multi-user.target操作命令：
- systemctl enable etcd
- systemctl start etcd